Final project_Advanced Modelling
Introduction
The objective of this assignment is to to explore two key areas of predictive modeling: classification and advanced regression, applied to student academic performance. In the classification section our goal is to predict the academic status (graduate, dropout, or enrolled) of students based on various socioeconomic, academic, and institutional factors. This could be helpful to identify early indicators of student success or risk of dropout. On its part, the regression analysis will focus on predicting the average grade of students during their first year of university, using the information available in our dataset.
In sum, by implementing statistical and machine learning approaches, we aim to determine which factors most influence student performance. For it, a publicly available dataset from the Instituto Politécnico de Portalegre, Portugal, was used. The dataset contains 4,424 records spanning academic years 2008/2009 to 2018/2019, integrating information from multiple sources, including the Academic Management System (AMS) and macroeconomic data from PORDATA. I consider it a valuable resource for educational analytics, and expect our findings to provide valuable insights for universities and policymakers to improve student retention and academic outcomes.
Load libraries
## Warning: package 'DataExplorer' was built under R version 4.3.3
## Warning: package 'ggplot2' was built under R version 4.3.3
## Warning: package 'forcats' was built under R version 4.3.2
## ── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
## ✔ dplyr 1.1.3 ✔ purrr 1.0.2
## ✔ forcats 1.0.0 ✔ stringr 1.5.0
## ✔ ggplot2 3.5.1 ✔ tibble 3.2.1
## ✔ lubridate 1.9.2 ✔ tidyr 1.3.0
## ── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
## ✖ dplyr::filter() masks stats::filter()
## ✖ dplyr::lag() masks stats::lag()
## ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors
## Warning: package 'plotly' was built under R version 4.3.2
##
## Attaching package: 'plotly'
##
## The following object is masked from 'package:ggplot2':
##
## last_plot
##
## The following object is masked from 'package:stats':
##
## filter
##
## The following object is masked from 'package:graphics':
##
## layout
## Loading required package: lattice
##
## Attaching package: 'caret'
##
## The following object is masked from 'package:purrr':
##
## lift
## Warning: package 'GGally' was built under R version 4.3.3
## Registered S3 method overwritten by 'GGally':
## method from
## +.gg ggplot2
## Warning: package 'ggeffects' was built under R version 4.3.3
##
## Attaching package: 'MASS'
##
## The following object is masked from 'package:plotly':
##
## select
##
## The following object is masked from 'package:dplyr':
##
## select
## Warning: package 'glmnet' was built under R version 4.3.3
## Loading required package: Matrix
##
## Attaching package: 'Matrix'
##
## The following objects are masked from 'package:tidyr':
##
## expand, pack, unpack
##
## Loaded glmnet 4.1-8
## Warning: package 'MLmetrics' was built under R version 4.3.3
##
## Attaching package: 'MLmetrics'
##
## The following objects are masked from 'package:caret':
##
## MAE, RMSE
##
## The following object is masked from 'package:base':
##
## Recall
## Warning: package 'rpart.plot' was built under R version 4.3.3
## Warning: package 'doParallel' was built under R version 4.3.3
## Loading required package: foreach
##
## Attaching package: 'foreach'
##
## The following objects are masked from 'package:purrr':
##
## accumulate, when
##
## Loading required package: iterators
## Loading required package: parallel
## Warning: package 'xgboost' was built under R version 4.3.3
##
## Attaching package: 'xgboost'
##
## The following object is masked from 'package:plotly':
##
## slice
##
## The following object is masked from 'package:dplyr':
##
## slice
## Warning: package 'olsrr' was built under R version 4.3.3
##
## Attaching package: 'olsrr'
##
## The following object is masked from 'package:MASS':
##
## cement
##
## The following object is masked from 'package:datasets':
##
## rivers
## Warning: package 'randomForest' was built under R version 4.3.3
## randomForest 4.7-1.1
## Type rfNews() to see new features/changes/bug fixes.
##
## Attaching package: 'randomForest'
##
## The following object is masked from 'package:dplyr':
##
## combine
##
## The following object is masked from 'package:ggplot2':
##
## margin
## Warning: package 'pdp' was built under R version 4.3.3
##
## Attaching package: 'pdp'
##
## The following object is masked from 'package:purrr':
##
## partial
Read dataset
## 'data.frame': 4424 obs. of 37 variables:
## $ Marital.status : int 1 1 1 1 2 2 1 1 1 1 ...
## $ Application.mode : int 17 15 1 17 39 39 1 18 1 1 ...
## $ Application.order : int 5 1 5 2 1 1 1 4 3 1 ...
## $ Course : int 171 9254 9070 9773 8014 9991 9500 9254 9238 9238 ...
## $ Daytime.evening.attendance. : int 1 1 1 1 0 0 1 1 1 1 ...
## $ Previous.qualification : int 1 1 1 1 1 19 1 1 1 1 ...
## $ Previous.qualification..grade. : num 122 160 122 122 100 ...
## $ Nacionality : int 1 1 1 1 1 1 1 1 62 1 ...
## $ Mother.s.qualification : int 19 1 37 38 37 37 19 37 1 1 ...
## $ Father.s.qualification : int 12 3 37 37 38 37 38 37 1 19 ...
## $ Mother.s.occupation : int 5 3 9 5 9 9 7 9 9 4 ...
## $ Father.s.occupation : int 9 3 9 3 9 7 10 9 9 7 ...
## $ Admission.grade : num 127 142 125 120 142 ...
## $ Displaced : int 1 1 1 1 0 0 1 1 0 1 ...
## $ Educational.special.needs : int 0 0 0 0 0 0 0 0 0 0 ...
## $ Debtor : int 0 0 0 0 0 1 0 0 0 1 ...
## $ Tuition.fees.up.to.date : int 1 0 0 1 1 1 1 0 1 0 ...
## $ Gender : int 1 1 1 0 0 1 0 1 0 0 ...
## $ Scholarship.holder : int 0 0 0 0 0 0 1 0 1 0 ...
## $ Age.at.enrollment : int 20 19 19 20 45 50 18 22 21 18 ...
## $ International : int 0 0 0 0 0 0 0 0 1 0 ...
## $ Curricular.units.1st.sem..credited. : int 0 0 0 0 0 0 0 0 0 0 ...
## $ Curricular.units.1st.sem..enrolled. : int 0 6 6 6 6 5 7 5 6 6 ...
## $ Curricular.units.1st.sem..evaluations. : int 0 6 0 8 9 10 9 5 8 9 ...
## $ Curricular.units.1st.sem..approved. : int 0 6 0 6 5 5 7 0 6 5 ...
## $ Curricular.units.1st.sem..grade. : chr "0.0" "14.0" "0.0" "13.428.571.428.571.400" ...
## $ Curricular.units.1st.sem..without.evaluations.: int 0 0 0 0 0 0 0 0 0 0 ...
## $ Curricular.units.2nd.sem..credited. : int 0 0 0 0 0 0 0 0 0 0 ...
## $ Curricular.units.2nd.sem..enrolled. : int 0 6 6 6 6 5 8 5 6 6 ...
## $ Curricular.units.2nd.sem..evaluations. : int 0 6 0 10 6 17 8 5 7 14 ...
## $ Curricular.units.2nd.sem..approved. : int 0 6 0 5 6 5 8 0 6 2 ...
## $ Curricular.units.2nd.sem..grade. : chr "0.0" "13.666.666.666.666.600" "0.0" "12.4" ...
## $ Curricular.units.2nd.sem..without.evaluations.: int 0 0 0 0 0 5 0 0 0 0 ...
## $ Unemployment.rate : num 10.8 13.9 10.8 9.4 13.9 16.2 15.5 15.5 16.2 8.9 ...
## $ Inflation.rate : num 1.4 -0.3 1.4 -0.8 -0.3 0.3 2.8 2.8 0.3 1.4 ...
## $ GDP : num 1.74 0.79 1.74 -3.12 0.79 -0.92 -4.06 -4.06 -0.92 3.51 ...
## $ Academic.status : chr "Dropout" "Graduate" "Dropout" "Graduate" ...
Data cleaning and preprocessing
The first thing I realize is that column names are long and complex, so I decided to make them more manageable by renaming them to shorter, more intuitive names. Here’s a brief description of some of the variables in our dataset, with the newly assigned names:
Demographic Features:
marital_status: Marital status of the student.gender: Gender of the student.age_enrollment: Age at enrollment.nationality: Nationality of the student.
Academic Features:
prev_qualification: Previous qualification.prev_grade: Grade from previous qualification.admission_grade: Admission grade.course: Course enrolled in.enrolled_1st_sem: Number of courses enrolled in the first semester.approved_1st_sem: Number of courses approved in the first semester.grade_1st_sem: Average grade in the first semester.enrolled_2nd_sem: Number of courses enrolled in the second semester.approved_2nd_sem: Number of courses approved in the second semester.grade_2nd_sem: Average grade in the second semester.academic_status: Academic status of the student.
Socioeconomic Features:
mother_qual: Mother’s qualification.father_qual: Father’s qualification.mother_occ: Mother’s occupation.father_occ: Father’s occupation.unemployment_rate: Unemployment rate.inflation_rate: Inflation rate.gdp: GDP.
# Original variable names
original_names <- names(data)
# Define new, shorter names
new_names <- c(
"marital_status", "app_mode", "app_order", "course", "attendance",
"prev_qualification", "prev_grade", "nationality", "mother_qual",
"father_qual", "mother_occ", "father_occ", "admission_grade",
"displaced", "special_needs", "debtor", "tuition_up_to_date",
"gender", "scholarship", "age_enrollment", "international",
"credits_1st_sem", "enrolled_1st_sem", "evaluations_1st_sem",
"approved_1st_sem", "grade_1st_sem", "no_eval_1st_sem",
"credits_2nd_sem", "enrolled_2nd_sem", "evaluations_2nd_sem",
"approved_2nd_sem", "grade_2nd_sem", "no_eval_2nd_sem",
"unemployment_rate", "inflation_rate", "gdp", "academic_status"
)
# Assign the new names to the data frame
names(data) <- new_names
# Check the updated variable names
names(data)## [1] "marital_status" "app_mode" "app_order"
## [4] "course" "attendance" "prev_qualification"
## [7] "prev_grade" "nationality" "mother_qual"
## [10] "father_qual" "mother_occ" "father_occ"
## [13] "admission_grade" "displaced" "special_needs"
## [16] "debtor" "tuition_up_to_date" "gender"
## [19] "scholarship" "age_enrollment" "international"
## [22] "credits_1st_sem" "enrolled_1st_sem" "evaluations_1st_sem"
## [25] "approved_1st_sem" "grade_1st_sem" "no_eval_1st_sem"
## [28] "credits_2nd_sem" "enrolled_2nd_sem" "evaluations_2nd_sem"
## [31] "approved_2nd_sem" "grade_2nd_sem" "no_eval_2nd_sem"
## [34] "unemployment_rate" "inflation_rate" "gdp"
## [37] "academic_status"
First insight into the data
We don’t have missing values, being the vast majority of our columns
continuous. However, most of them aren’t in the appropriate format. We
first convert categorical variables into factor.
categorical_vars <- c(
"marital_status", "app_mode", "course", "attendance",
"prev_qualification", "nationality", "mother_qual",
"father_qual", "mother_occ", "father_occ",
"displaced", "special_needs", "debtor", "tuition_up_to_date",
"gender", "scholarship", "international", "academic_status")
data[categorical_vars] <- lapply(data[categorical_vars], as.factor)There were two columns, grade_1st_sem and grade_1st_sem, that caused issues because they contained decimal numbers with multiple periods. In R, such formats are not valid numeric values, causing NA values when attempting to convert them directly to numeric. To fix this, we cleaned the data by extracting only the first decimal point and the following digit, then converted the cleaned strings to numeric values and rounded them to one decimal place. This ensures the data is properly formatted for analysis. However, we also identified and removed records where grades exceeded the maximum possible value of 20, as specified by the dataset creators. This step ensures data integrity and aligns with the established grading scale of 0 to 20.
# Function to clean and round to one decimal place
clean_and_round <- function(column) {
# Extract the first decimal point and the following digit
column <- gsub("(\\d+\\.\\d).*", "\\1", column)
# Convert to numeric and round to one decimal place
round(as.numeric(column), digits = 1)
}
# Apply the function to the problematic columns
data$grade_1st_sem <- clean_and_round(data$grade_1st_sem)
data$grade_2nd_sem <- clean_and_round(data$grade_2nd_sem)
# Filter records where grade_1st_sem is less than or equal to 20
data <- data[data$grade_1st_sem <= 20, ]
data <- data[data$grade_2nd_sem <= 20, ]Then, we convert binary variables into numeric.
# List of binary variables
binary_vars <- c("attendance", "displaced", "special_needs", "debtor",
"tuition_up_to_date", "gender", "scholarship", "international")
# Converting binary variables into numeric
data[binary_vars] <- lapply(data[binary_vars],
function(x) as.numeric(as.character(x)))
# Verifying the result
str(data[, binary_vars])## 'data.frame': 4321 obs. of 8 variables:
## $ attendance : num 1 1 1 1 0 0 1 1 1 1 ...
## $ displaced : num 1 1 1 1 0 0 1 1 0 1 ...
## $ special_needs : num 0 0 0 0 0 0 0 0 0 0 ...
## $ debtor : num 0 0 0 0 0 1 0 0 0 1 ...
## $ tuition_up_to_date: num 1 0 0 1 1 1 1 0 1 0 ...
## $ gender : num 1 1 1 0 0 1 0 1 0 0 ...
## $ scholarship : num 0 0 0 0 0 0 1 0 1 0 ...
## $ international : num 0 0 0 0 0 0 0 0 1 0 ...
# Renaming
colnames(data)[colnames(data) == "gender"] <- "gender_Male" # 1 male 0 female
colnames(data)[colnames(data) == "attendance"] <- "attendance_Daytime"
# 1 daytime 2 evening attendanceNext, we transform factor variables into numeric.
Marital status: we realized the vast majority are single, so we will convert it into a dummy
##
## 1 2 3 4 5 6
## 3824 372 4 90 25 6
data <- data %>%
mutate(single = ifelse(marital_status == 1, 1, 0)) %>% #If single 1, if not, 0
dplyr::select(-marital_status)Application mode: we transform this variable into a set of more specific binary variables, each of them representing a specific and broad category of admission.
##
## 1 2 5 7 10 15 16 17 18 26 27 39 42 43 44 51
## 1666 3 16 137 10 30 38 845 116 1 1 778 73 299 213 59
## 53 57
## 35 1
data <- data %>%
mutate(
app_regular = ifelse(app_mode %in% c(1, 17, 18), 1, 0), # Regular Admission
app_special = ifelse(app_mode %in% c(5, 15, 16, 2, 10, 26, 27), 1, 0),
# Special Cases (International, Ordinances, Regional)
app_over23 = ifelse(app_mode == 39, 1, 0), # Over 23 Years Old
app_transfer_coursechange = ifelse(app_mode %in% c(42, 43, 51, 57), 1, 0),
# Transfers & Course Changes
app_previous_highered = ifelse(app_mode %in% c(7, 44, 53), 1, 0)
# Holders of Previous Higher Education Degrees
) %>%
dplyr::select(-app_mode)Course: we do the same for course, grouping them by fields.
##
## 33 171 8014 9003 9070 9085 9119 9130 9147 9238 9254 9500 9556 9670 9773 9853
## 12 215 213 210 226 337 170 140 380 355 252 684 69 268 331 192
## 9991
## 267
data <- data %>%
mutate(
engineering_tech = ifelse(course %in% c(9119), 1, 0),
# Engineering & Technology
health_sciences = ifelse(course %in% c(9085, 9500, 9556), 1, 0),
# Health Sciences
management_business = ifelse(course %in% c(9147, 9991), 1, 0),
# Management & Business
social_sciences_humanities = ifelse(course %in% c(8014, 9238, 9853), 1, 0),
# Social Sciences & Humanities
agriculture_env = ifelse(course %in% c(33, 9003, 9130), 1, 0),
# Agriculture & Environmental Sciences
arts_media_comm = ifelse(course %in% c(171, 9070, 9670, 9773), 1, 0),
# Arts, Media & Communication
tourism_hospitality = ifelse(course == 9254, 1, 0)
# Tourism & Hospitality
) %>%
dplyr::select(-course)Previous qualification: the vast majority had previous secondary studies so we convert it into a dummy.
##
## 1 2 3 4 5 6 9 10 12 14 15 19 38 39 40 42
## 3624 21 124 8 1 16 11 4 45 1 2 159 6 219 38 36
## 43
## 6
data <- data %>%
mutate(secondary_education = ifelse(prev_qualification == 1, 1, 0))%>%
dplyr::select(-prev_qualification)Nationality: if Portuguese 1, if not 0
##
## 1 2 6 11 13 14 17 21 22 24 25 26 32 41 62 100
## 4212 2 12 3 1 1 1 2 13 5 2 14 1 38 2 3
## 101 103 105 108 109
## 2 3 2 1 1
Several studies in educational sociology indicate that maternal education has a stronger influence on student success than paternal education. Empirical evidence from researches such as Davis-Kean (2005) and Fan & Chen (2001) supports this, showing that children of highly educated mothers are more likely to graduate and achieve higher academic performance. Given these findings, variable mother_qual was retained and recategorized, while father_qual and father_occ were removed to limit the number of variables and prioritize the most impactful predictors.
##
## 1 2 3 4 5 6 9 10 11 12 14 18 19 22 26 27
## 1036 83 426 46 21 4 8 3 3 40 2 1 930 1 1 1
## 29 30 34 35 36 37 38 39 40 41 42 43 44
## 3 3 128 3 3 993 550 8 9 6 4 4 1
data <- data %>%
mutate(
mother_secondary = ifelse(mother_qual %in% c(1, 9, 10, 12, 14, 18), 1, 0),
mother_higher = ifelse(mother_qual %in% c(2, 3, 4, 5, 6, 40, 41, 42, 43, 44),
1, 0),
mother_basic = ifelse(mother_qual %in% c(19, 22, 26, 27, 29, 30, 37, 38, 39),
1, 0),
mother_illiterate_other = ifelse(mother_qual %in% c(34, 35, 36, 11), 1, 0)
)%>%
dplyr::select(-mother_qual, -father_qual, -father_occ)Mother occupation: we simplified and grouped similar occupations together.
##
## 0 1 2 3 4 5 6 7 8 9 10 90 99 122 123 125
## 142 102 310 341 787 517 91 265 36 1544 4 70 17 2 7 1
## 131 132 134 141 143 144 151 152 153 171 173 175 191 192 193 194
## 1 3 4 8 3 6 3 2 2 1 1 5 26 5 4 11
data <- data %>%
mutate(
mother_no_working = ifelse(mother_occ %in% c(0, 90, 99), 1, 0),
mother_high_position = ifelse(mother_occ %in% c(1, 2, 122, 123, 125), 1, 0),
mother_technical_admin = ifelse(mother_occ %in% c(3, 131, 132, 134, 141, 143,
144), 1, 0),
mother_services_sales = ifelse(mother_occ %in% c(5, 151, 152, 153, 194), 1, 0),
mother_agriculture = ifelse(mother_occ %in% c(6, 192), 1, 0),
mother_industry = ifelse(mother_occ %in% c(7, 8, 171, 173, 175, 193), 1, 0),
mother_cleaning = ifelse(mother_occ %in% c(191), 1, 0),
mother_unskilled = ifelse(mother_occ %in% c(9), 1, 0)
) %>%
dplyr::select(-mother_occ)As we are interested on early prediction, we aim to identify students at risk of dropping out as soon as possible, allowing institutions to implement timely interventions. Unlike models that rely on long-term academic data, our approach emphasizes early university performance. Consequently, we will mantain first-semester information, while excluding second-semester results, ensuring that predictions are made in the initial stages of higher education.
Part 1: Classification
Exploratory Data Analysis
Exploring the target variable
##
## Dropout Enrolled Graduate
## 0.3267762 0.1821338 0.4910900
# Distribution of academic status
ggplot(data, aes(x = academic_status, fill = academic_status)) +
geom_bar() +
labs(title = "Distribution of Academic Status", x = "Academic Status",
y = "Count")+
theme(legend.position = "none")
The dataset shows a slight class imbalance, with Graduates (49.1%) being
the most frequent category, followed by Dropouts (32.7%) and Enrolled
students (18.2%). Our primary goal is to classify students into all
three categories rather than focusing on a binary distinction, since
this approach aligns better with the objectives of the course, allowing
us to explore multi-class classification challenges. However, we
acknowledge that the Enrolled category is the smallest and may be harder
for the model to predict accurately. Since our main interest is in
identifying students at risk of dropping out, misclassifications in the
Dropout category will be considered at the highest cost, while errors in
the Enrolled or Graduate categories will be of lesser concern.
Exploring some relations with our target variable:
- Academic performance predictors
# Admission Grade vs. Academic Status
ggplot(data, aes(x = academic_status, y = admission_grade, fill = academic_status)) +
geom_boxplot() +
labs(title = "Admission Grade Distribution by Academic Status",
x = "Academic Status", y = "Admission Grade") +
theme_minimal()+
theme(legend.position = "none")# First Semester Grade vs. Academic Status
ggplot(data, aes(x = academic_status, y = grade_1st_sem, fill = academic_status)) +
geom_boxplot() +
labs(title = "First Semester Grade Distribution by Academic Status",
x = "Academic Status", y = "First Semester Grade") +
theme_minimal()+
theme(legend.position = "none")# Previous Grade vs. Academic Status
ggplot(data, aes(x = academic_status, y = prev_grade, fill = academic_status)) +
geom_boxplot() +
labs(title = "Previous Grade Distribution by Academic Status",
x = "Academic Status", y = "Previous Grade") +
theme_minimal()+
theme(legend.position = "none")
While previous and admission grades do not seem to show significant
differences between the groups, the most striking pattern emerges with
the first-semester grades. Graduates tend to have the highest
first-semester grades with a compact distribution, while enrolled
students have slightly lower but similar medians. In contrast, dropouts
exhibit a much wider dispersion, with many students having grades close
to zero. This suggests that early academic performance plays a crucial
role in determining student outcomes, as those who struggle in their
first semester are at a much higher risk of dropping out. The sharp
contrast in grade distributions indicates that first-semester
performance could serve as a strong early indicator for academic success
or failure.
- Socioeconomic factors
# Gender vs Academic Status
ggplot(data, aes(x = factor(gender_Male, labels = c("Female", "Male")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
theme_minimal() +
labs(title = "Gender vs Academic Status", y = "Proportion",
x = "Gender", fill = "Academic Status")# Scholarship vs. Academic Status
ggplot(data, aes(x = factor(scholarship, labels = c("No", "Yes")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
labs(title = "Scholarship Holders by Academic Status",
x = "Scholarship", y = "Proportion", fill = "Academic Status") +
theme_minimal()# Tuition Up-to-Date vs. Academic Status
ggplot(data, aes(x = factor(tuition_up_to_date, labels = c("No", "Yes")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
labs(title = "Tuition Payment Status by Academic Status",
x = "Tuition up-to-date", y = "Proportion", fill = "Academic Status") +
theme_minimal()# Mother's Education vs. Academic Status
ggplot(data, aes(x = factor(mother_secondary, labels = c("No", "Yes")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
labs(title = "Mother's Secondary Education & Academic Status",
x = "Mother Secondary Educ.", y = "Proportion", fill = "Academic Status") +
theme_minimal()
Male students tend to dropout more than female and most students who
receive a scholarship tend to graduate, while only a small minority of
them dropout. On the other hand, among students who do not have a
scholarship, the proportions of graduates and dropouts are more
balanced.
Additionally, a significant pattern emerges regarding tuition payment status: the majority of students who do not have their tuition up to date appear in the dropout category. Since nearly 80% of students in this situation dropout, this variable might be too dominant and could strongly influence the model. It raises the question of whether we should consider removing it, as it could act more as a direct proxy for dropout rather than an explanatory feature, potentially leading to overly simplistic predictions. We finally decide to remove it.
- Institutional & enrollment variables
# Application Mode vs. Academic Status
ggplot(data, aes(x = factor(app_regular, labels = c("No", "Yes")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
labs(title = "Application Mode & Academic Status",
x = "Regular Application", y = "Proportion", fill = "Academic Status") +
theme_minimal()# Study Program vs. Academic Status
ggplot(data, aes(x = factor(health_sciences, labels = c("No", "Yes")),
fill = as.factor(academic_status))) +
geom_bar(position = "fill") +
labs(title = "Health Sciences Students & Academic Status",
x = "Health Sciences", y = "Proportion", fill = "Academic Status") +
theme_minimal()
People that entered university through the regular modalities are more
prone to graduate, compared those who applied by other methods.
Moreover, there seems to be a slightly higher proportion of graduates
among health sciences students compared to those in other fields of
study, maybe due to the vocational nature or compromise these degrees
are usually associated with.
Correlation analysis and feature selection
In this section we aim to analyze the correlations between numerical variables in the dataset.
# Select only numeric variables
numeric_data <- data[, sapply(data, is.numeric)]
# Compute the correlation matrix
cor_matrix <- cor(numeric_data, use = "complete.obs")
# Install and load the corrplot package
library(corrplot)## corrplot 0.92 loaded
# Plot the correlation matrix
corrplot(cor_matrix, method = "color", type = "upper", tl.col = "black",
tl.srt = 45)
The correlation matrix is converted into a data frame to identify highly
correlated variable pairs. Pairs with a correlation coefficient above
0.7 (excluding self-correlations) are extracted and sorted by absolute
correlation value. This helps detect potential multicollinearity, which
may indicate redundant features that should be removed or adjusted to
improve model performance.
# Convert the correlation matrix to a data frame
cor_df <- as.data.frame(as.table(cor_matrix))
# Filter pairs with correlation >= 0.8 (excluding self-correlations)
high_cor_pairs <- subset(cor_df, abs(Freq) >= 0.7 & Var1 != Var2)
# Sort by absolute correlation value
high_cor_pairs <- high_cor_pairs[order(-abs(high_cor_pairs$Freq)), ]
# Display the highly correlated pairs
print(high_cor_pairs)## Var1 Var2 Freq
## 505 portuguese international -1.0000000
## 1609 international portuguese -1.0000000
## 530 enrolled_1st_sem credits_1st_sem 0.7762595
## 576 credits_1st_sem enrolled_1st_sem 0.7762595
## 579 approved_1st_sem enrolled_1st_sem 0.7673513
## 671 enrolled_1st_sem approved_1st_sem 0.7673513
We will eliminate “enrolled_1st_sem” and “portuguese” to reduce noise and redundancy in the dataset. “Enrolled_1st_sem” is highly correlated with both “credits_1st_sem” and “approved_1st_sem,” meaning it does not add much independent information. Similarly, “portuguese” and “international” are perfectly negatively correlated, making one of them redundant.
Statistical classification
As part of the statistical classification approach, we will explore several models, including multinomial logistic regression, Linear Discriminant Analysis, and Quadratic Discriminant Analysis, which fall under the umbrella of Bayesian- inspired techniques. These models will help us understand the relationships between predictor variables and academic status.
Multinomial logistic regression model
To classify students into Dropout, Enrolled, or Graduate, we begin with a multinomial logistic regression model. This method allows us to model categorical outcomes with more than two classes by estimating the probability of each student belonging to one of the three academic status categories.
First, we split the data into an 80% training set and a 20% testing set to ensure that our model is evaluated on unseen data.
# Data partition
set.seed(123)
in_train <- createDataPartition(data$academic_status, p = 0.8, list = FALSE)
training <- data[in_train,]
testing <- data[-in_train,]
nrow(training)## [1] 3458
## [1] 863
We will begin with a theory-driven approach, selecting variables based on domain knowledge and our exploratory data analysis. This will gives us an initial sense of which variables are the most relevant and whether they behave as expected. Additionally, we exponentiate the model coefficients to interpret their effects in terms of odds ratios, helping us understand the impact of different features on the likelihood of a student belonging to each academic category.
# Train the multinomial model with selected predictors
multinom.model <- multinom(academic_status ~ age_enrollment + gender_Male +
scholarship + prev_grade + admission_grade +
app_regular + app_over23 + engineering_tech
+ health_sciences + social_sciences_humanities
+ arts_media_comm + unemployment_rate + gdp
+ grade_1st_sem + mother_secondary + mother_higher
+ mother_basic,
data = training)## # weights: 57 (36 variable)
## initial value 3799.001294
## iter 10 value 3177.076928
## iter 20 value 3036.693240
## iter 30 value 2905.996892
## iter 40 value 2880.282411
## final value 2880.268210
## converged
## Call:
## multinom(formula = academic_status ~ age_enrollment + gender_Male +
## scholarship + prev_grade + admission_grade + app_regular +
## app_over23 + engineering_tech + health_sciences + social_sciences_humanities +
## arts_media_comm + unemployment_rate + gdp + grade_1st_sem +
## mother_secondary + mother_higher + mother_basic, data = training)
##
## Coefficients:
## (Intercept) age_enrollment gender_Male scholarship prev_grade
## Enrolled -2.391291 -0.040412702 -0.2898622 0.486967 -0.0008075199
## Graduate -5.944324 -0.008016376 -0.4469380 1.379733 0.0036341924
## admission_grade app_regular app_over23 engineering_tech
## Enrolled 0.001428765 -0.07212635 -0.1717280 0.5485767
## Graduate 0.019440026 0.39605864 -0.5122709 -1.4641001
## health_sciences social_sciences_humanities arts_media_comm
## Enrolled 0.1804339 -0.0657495 -0.09346135
## Graduate 0.6840811 0.4495303 0.40090905
## unemployment_rate gdp grade_1st_sem mother_secondary
## Enrolled -0.08875476 -0.0397340995 0.1264997 2.684212
## Graduate -0.02375023 0.0002967505 0.1972951 1.293041
## mother_higher mother_basic
## Enrolled 3.031143 2.624385
## Graduate 1.395557 1.324164
##
## Std. Errors:
## (Intercept) age_enrollment gender_Male scholarship prev_grade
## Enrolled 0.9482232 0.010904433 0.1201398 0.1579744 0.005005757
## Graduate 0.6988920 0.008490139 0.1034572 0.1286428 0.004278093
## admission_grade app_regular app_over23 engineering_tech
## Enrolled 0.004568172 0.1454873 0.1817738 0.2372268
## Graduate 0.003897892 0.1288979 0.1603113 0.3697414
## health_sciences social_sciences_humanities arts_media_comm
## Enrolled 0.1531115 0.1717706 0.1497293
## Graduate 0.1327866 0.1425631 0.1282734
## unemployment_rate gdp grade_1st_sem mother_secondary
## Enrolled 0.02183994 0.02494637 0.01099843 0.6129159
## Graduate 0.01923843 0.02209633 0.01042007 0.3093812
## mother_higher mother_basic
## Enrolled 0.6194661 0.6055511
## Graduate 0.3226798 0.2970804
##
## Residual Deviance: 5760.536
## AIC: 5832.536
## (Intercept) age_enrollment gender_Male scholarship prev_grade
## Enrolled 0.091511462 0.9603930 0.7483667 1.627373 0.9991928
## Graduate 0.002620674 0.9920157 0.6395836 3.973842 1.0036408
## admission_grade app_regular app_over23 engineering_tech
## Enrolled 1.00143 0.9304133 0.8422083 1.730788
## Graduate 1.01963 1.4859565 0.5991335 0.231286
## health_sciences social_sciences_humanities arts_media_comm
## Enrolled 1.197737 0.9363654 0.9107732
## Graduate 1.981950 1.5675758 1.4931815
## unemployment_rate gdp grade_1st_sem mother_secondary
## Enrolled 0.9150700 0.9610449 1.134849 14.646656
## Graduate 0.9765296 1.0002968 1.218103 3.643852
## mother_higher mother_basic
## Enrolled 20.720912 13.796090
## Graduate 4.037222 3.759041
The multinomial model results highlight several key factors influencing student outcomes. Scholarship holders are significantly more likely to graduate or stay enrolled, reinforcing the role of financial support in academic success. Previous and admission grades have a minor but positive effect, while maternal education strongly correlates with higher enrollment and graduation rates. Engineering students are less likely to graduate compared to those in health sciences, potentially reflecting differences in program difficulty. The unemployment rate slightly decreases enrollment likelihood but has minimal impact on graduation.
set.seed(123)
# Predict on the training set
train_preds_multinom <- predict(multinom.model, newdata = training)
# Compute confusion matrix for the training set
conf_matrix_train <- confusionMatrix(train_preds_multinom,
training$academic_status)
print(conf_matrix_train)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 685 163 210
## Enrolled 28 43 17
## Graduate 417 424 1471
##
## Overall Statistics
##
## Accuracy : 0.6359
## 95% CI : (0.6196, 0.652)
## No Information Rate : 0.491
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.358
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.6062 0.06825 0.8663
## Specificity 0.8398 0.98409 0.5222
## Pos Pred Value 0.6474 0.48864 0.6362
## Neg Pred Value 0.8146 0.82582 0.8019
## Prevalence 0.3268 0.18219 0.4910
## Detection Rate 0.1981 0.01243 0.4254
## Detection Prevalence 0.3060 0.02545 0.6686
## Balanced Accuracy 0.7230 0.52617 0.6942
The confusion matrix shows an overall accuracy of 63.6%, performing well for predicting graduates (86.6% sensitivity) and moderately for dropouts (60.6%). However, it struggles with enrolled students (6.8% sensitivity), likely due to class imbalance.
# Predict on the test set
test_preds_multinom <- predict(multinom.model, newdata = testing)
# Compute confusion matrix for the test set
conf_matrix_test <- confusionMatrix(test_preds_multinom, testing$academic_status)
print(conf_matrix_test)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 176 45 55
## Enrolled 13 7 5
## Graduate 93 105 364
##
## Overall Statistics
##
## Accuracy : 0.6338
## 95% CI : (0.6007, 0.6661)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.3579
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.6241 0.044586 0.8585
## Specificity 0.8279 0.974504 0.5490
## Pos Pred Value 0.6377 0.280000 0.6477
## Neg Pred Value 0.8194 0.821002 0.8007
## Prevalence 0.3268 0.181924 0.4913
## Detection Rate 0.2039 0.008111 0.4218
## Detection Prevalence 0.3198 0.028969 0.6512
## Balanced Accuracy 0.7260 0.509545 0.7037
Overall, the model is performing well for dropout and graduate predictions but has difficulty identifying enrolled students due probably to class imbalance and underrepresentation.
We are now going to visualize two graphs useful for understanding how predictor variables impact the likelihood of a student being classified into the different academic statuses, allowing us to interpret the model’s behavior more intuitively.
# Visualizar el impacto de una variable
gg.pred <- ggpredict(multinom.model, terms = "mother_secondary")
plot(gg.pred)# Visualizar el impacto de una variable
gg.pred <- ggpredict(multinom.model, terms = "grade_1st_sem")## Data were 'prettified'. Consider using `terms="grade_1st_sem [all]"` to
## get smooth plots.
The first graph suggests that having a mother with secondary education
positively influences a student’s chances of continuing their education
and graduating, aligning with findings from sociological studies on
parental education. The second one highlights how first semester grade
plays a key role in predicting a student’s academic status, with lower
grades correlating with higher dropout probabilities, and higher grades
correlating with higher graduation probabilities.
Lasso regularized multinomial model
set.seed(123)
# Preparing data for glmnet
y_train <- as.numeric(training$academic_status) - 1
y_test <- as.numeric(testing$academic_status) - 1
# Matrix creation
X_train <- model.matrix(academic_status ~ ., data = training)[,-1]
X_test <- model.matrix(academic_status ~ ., data = testing)[,-1]We train the model
set.seed(123)
# Train Lasso model
lasso_model <- cv.glmnet(X_train, y_train,
family = "multinomial",
alpha = 1, # (L1 penalization)
type.multinomial = "grouped")
# Select best lambda
best_lambda <- lasso_model$lambda.min
print(best_lambda)## [1] 0.004189825
We make predictions on the training set
lasso_preds <- predict(lasso_model, newx = X_test, s = "lambda.min",
type = "response")
# We select the highest probablity class
lasso_preds <- apply(lasso_preds, 1, which.max) - 1
# Converting to factor
lasso_preds <- as.factor(lasso_preds + 1)
y_test_factor <- as.factor(y_test + 1)
# Evalutation
conf_matrix_lasso <- confusionMatrix(lasso_preds, y_test_factor)
print(conf_matrix_lasso)## Confusion Matrix and Statistics
##
## Reference
## Prediction 1 2 3
## 1 224 52 37
## 2 19 25 13
## 3 39 80 374
##
## Overall Statistics
##
## Accuracy : 0.7219
## 95% CI : (0.6907, 0.7516)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.5277
##
## Mcnemar's Test P-Value : 9.706e-14
##
## Statistics by Class:
##
## Class: 1 Class: 2 Class: 3
## Sensitivity 0.7943 0.15924 0.8821
## Specificity 0.8468 0.95467 0.7289
## Pos Pred Value 0.7157 0.43860 0.7586
## Neg Pred Value 0.8945 0.83623 0.8649
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2596 0.02897 0.4334
## Detection Prevalence 0.3627 0.06605 0.5713
## Balanced Accuracy 0.8206 0.55695 0.8055
The confusion matrix results show that the model has an overall accuracy of 72.07%, with a Kappa value of 0.5256, indicating moderate agreement between predicted and actual values. The model performs better at predicting Dropout (Class 1) and Graduate (Class 3) statuses, with accuracy rates of 79.43% and 88.21% respectively, while the model struggles more with predicting the Enrolled (Class 2) status, with only 15.29% sensitivity.
## $`0`
## 46 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.713237499
## app_order 0.074965630
## attendance_Daytime 0.081770616
## prev_grade .
## admission_grade -0.005367366
## displaced 0.001255805
## special_needs .
## debtor 0.803788614
## gender_Male 0.192317793
## scholarship -0.485826131
## age_enrollment 0.021278702
## international .
## credits_1st_sem 0.244667222
## evaluations_1st_sem 0.033408020
## approved_1st_sem -0.381287686
## grade_1st_sem -0.003900635
## no_eval_1st_sem -0.021975143
## unemployment_rate 0.044492577
## inflation_rate 0.004378908
## gdp 0.004117625
## single 0.027663899
## app_regular 0.008724782
## app_special -0.392875174
## app_over23 0.160284497
## app_transfer_coursechange .
## app_previous_highered -0.015669453
## engineering_tech -0.050259634
## health_sciences .
## management_business -0.086659816
## social_sciences_humanities 0.024904787
## agriculture_env 0.088471695
## arts_media_comm -0.044605126
## tourism_hospitality 0.329524457
## secondary_education -0.085815503
## mother_secondary .
## mother_higher -0.134894895
## mother_basic 0.009704527
## mother_illiterate_other 0.619934215
## mother_no_working 0.654605697
## mother_high_position .
## mother_technical_admin -0.012273604
## mother_services_sales 0.045981131
## mother_agriculture -0.227116850
## mother_industry 0.004761837
## mother_cleaning -2.044171816
## mother_unskilled -0.045483148
##
## $`1`
## 46 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) 0.5804972035
## app_order -0.0608636633
## attendance_Daytime 0.0014453782
## prev_grade .
## admission_grade -0.0030127380
## displaced -0.0006902969
## special_needs .
## debtor 0.0595532251
## gender_Male -0.0258818504
## scholarship -0.1012044168
## age_enrollment -0.0140366944
## international .
## credits_1st_sem -0.0884042928
## evaluations_1st_sem 0.0894775719
## approved_1st_sem -0.0688167494
## grade_1st_sem 0.0041652024
## no_eval_1st_sem -0.0142062241
## unemployment_rate -0.0445904518
## inflation_rate 0.0070865919
## gdp 0.0010018119
## single -0.0354694685
## app_regular -0.1122477080
## app_special 0.2288139878
## app_over23 -0.0886461914
## app_transfer_coursechange .
## app_previous_highered 0.0225367201
## engineering_tech 0.7836092224
## health_sciences .
## management_business 0.1969500845
## social_sciences_humanities -0.0747001882
## agriculture_env -0.1582565682
## arts_media_comm 0.0134068565
## tourism_hospitality -0.2106812790
## secondary_education 0.2118050807
## mother_secondary .
## mother_higher 0.1752861196
## mother_basic -0.0141760523
## mother_illiterate_other -0.3262784999
## mother_no_working -1.2158892278
## mother_high_position .
## mother_technical_admin 0.0607512666
## mother_services_sales 0.0168180650
## mother_agriculture 0.0009180042
## mother_industry 0.0248528326
## mother_cleaning 1.0822797216
## mother_unskilled -0.0316269807
##
## $`2`
## 46 x 1 sparse Matrix of class "dgCMatrix"
## 1
## (Intercept) -1.293735e+00
## app_order -1.410197e-02
## attendance_Daytime -8.321599e-02
## prev_grade .
## admission_grade 8.380104e-03
## displaced -5.655081e-04
## special_needs .
## debtor -8.633418e-01
## gender_Male -1.664359e-01
## scholarship 5.870305e-01
## age_enrollment -7.242008e-03
## international .
## credits_1st_sem -1.562629e-01
## evaluations_1st_sem -1.228856e-01
## approved_1st_sem 4.501044e-01
## grade_1st_sem -2.645671e-04
## no_eval_1st_sem 3.618137e-02
## unemployment_rate 9.787526e-05
## inflation_rate -1.146550e-02
## gdp -5.119437e-03
## single 7.805570e-03
## app_regular 1.035229e-01
## app_special 1.640612e-01
## app_over23 -7.163831e-02
## app_transfer_coursechange .
## app_previous_highered -6.867267e-03
## engineering_tech -7.333496e-01
## health_sciences .
## management_business -1.102903e-01
## social_sciences_humanities 4.979540e-02
## agriculture_env 6.978487e-02
## arts_media_comm 3.119827e-02
## tourism_hospitality -1.188432e-01
## secondary_education -1.259896e-01
## mother_secondary .
## mother_higher -4.039122e-02
## mother_basic 4.471526e-03
## mother_illiterate_other -2.936557e-01
## mother_no_working 5.612835e-01
## mother_high_position .
## mother_technical_admin -4.847766e-02
## mother_services_sales -6.279920e-02
## mother_agriculture 2.261988e-01
## mother_industry -2.961467e-02
## mother_cleaning 9.618921e-01
## mother_unskilled 7.711013e-02
Financial aspects play a crucial role, with debtor status increasing dropout likelihood, while scholarships significantly enhance graduation prospects. Maternal education emerges as a powerful predictor across all outcomes, with higher levels generally associated with better student performance. The type of application (regular vs. special) shows some impact on enrollment status. Interestingly, maternal employment status also influences graduation rates, with students whose mothers are not working or are in cleaning occupations showing higher graduation probabilities.
Making predictions on the testing set
# Make predictions on the testing set
lasso_preds_test <- predict(lasso_model, newx = X_test, s = "lambda.min",
type = "response")
# Since glmnet returns probabilities, we select the class with the highest probability
lasso_preds_test <- apply(lasso_preds_test, 1, which.max) - 1
# Convert the predictions and the test labels to factors for confusion matrix comparison
lasso_preds_test <- as.factor(lasso_preds_test + 1)
y_test_factor <- as.factor(y_test + 1)
# Evaluate the model performance with confusion matrix
conf_matrix_lasso_test <- confusionMatrix(lasso_preds_test, y_test_factor)
# Print confusion matrix
print(conf_matrix_lasso_test)## Confusion Matrix and Statistics
##
## Reference
## Prediction 1 2 3
## 1 224 52 37
## 2 19 25 13
## 3 39 80 374
##
## Overall Statistics
##
## Accuracy : 0.7219
## 95% CI : (0.6907, 0.7516)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.5277
##
## Mcnemar's Test P-Value : 9.706e-14
##
## Statistics by Class:
##
## Class: 1 Class: 2 Class: 3
## Sensitivity 0.7943 0.15924 0.8821
## Specificity 0.8468 0.95467 0.7289
## Pos Pred Value 0.7157 0.43860 0.7586
## Neg Pred Value 0.8945 0.83623 0.8649
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2596 0.02897 0.4334
## Detection Prevalence 0.3627 0.06605 0.5713
## Balanced Accuracy 0.8206 0.55695 0.8055
We see how accuracy is better than in the previous one (72.07%) and sensitivity for the dropout class is better too (79.43%).
# Extracting number of selected variables
lasso_coefs_matrix <- as.matrix(lasso_coefs[[3]])
non_zero_coefs <- sum(lasso_coefs_matrix != 0)
print(non_zero_coefs)## [1] 39
We realize lasso performs better but uses too many variables. Taking advantage of its information, we will create a new multinomial logistic regression model, just using 13 variables.
Seccond attempt multinomial logistic regression model
set.seed(123)
in_train <- createDataPartition(data$academic_status, p = 0.8, list = FALSE)
training <- data[in_train,]
testing <- data[-in_train,]
# Train the new multinomial model with selected predictors
multinom.model2 <- multinom(academic_status ~ mother_cleaning + debtor +
mother_no_working +mother_illiterate_other +
tourism_hospitality +credits_1st_sem +gender_Male +
app_over23 +mother_higher + mother_agriculture +
approved_1st_sem +app_special +scholarship,
data = training)## # weights: 45 (28 variable)
## initial value 3799.001294
## iter 10 value 2847.169808
## iter 20 value 2531.643284
## iter 30 value 2499.446402
## iter 40 value 2497.830449
## iter 50 value 2497.757810
## final value 2497.757708
## converged
# Predict on the training set
train_preds_multinom2 <- predict(multinom.model2, newdata = training)
# Compute confusion matrix for the training set
conf_matrix_train2 <- confusionMatrix(train_preds_multinom2,
training$academic_status)
print(conf_matrix_train2)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 868 232 125
## Enrolled 10 26 10
## Graduate 252 372 1563
##
## Overall Statistics
##
## Accuracy : 0.7105
## 95% CI : (0.6951, 0.7256)
## No Information Rate : 0.491
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.4933
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.7681 0.041270 0.9205
## Specificity 0.8466 0.992928 0.6455
## Pos Pred Value 0.7086 0.565217 0.7147
## Neg Pred Value 0.8827 0.822978 0.8938
## Prevalence 0.3268 0.182186 0.4910
## Detection Rate 0.2510 0.007519 0.4520
## Detection Prevalence 0.3543 0.013302 0.6324
## Balanced Accuracy 0.8074 0.517099 0.7830
# Predict on the test set
test_preds_multinom2 <- predict(multinom.model2, newdata = testing)
# Compute confusion matrix for the test set
conf_matrix_test2 <- confusionMatrix(test_preds_multinom2,
testing$academic_status)
print(conf_matrix_test2)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 224 61 42
## Enrolled 2 3 1
## Graduate 56 93 381
##
## Overall Statistics
##
## Accuracy : 0.7045
## 95% CI : (0.6728, 0.7348)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.4845
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.7943 0.019108 0.8986
## Specificity 0.8227 0.995751 0.6606
## Pos Pred Value 0.6850 0.500000 0.7189
## Neg Pred Value 0.8918 0.820303 0.8709
## Prevalence 0.3268 0.181924 0.4913
## Detection Rate 0.2596 0.003476 0.4415
## Detection Prevalence 0.3789 0.006952 0.6141
## Balanced Accuracy 0.8085 0.507429 0.7796
These results show the performance of the multinomial model on both the training and testing sets. We see a 70.45% overall accuracy in the testing set, and 79.43% sensitivity and 82.27% specificity for the dropout category. We prefer this last model due to its simplicity and interpretability with a comparable performance.
Linear Discriminant Analysis (LDA)
Now, we will apply LDA to predict the academic status of students. We will use the selected variables based on previous analysis and the results from our prior models to train the LDA model.
set.seed(123)
lda_model <- lda(academic_status ~ mother_cleaning+debtor +mother_no_working +
mother_illiterate_other + tourism_hospitality +credits_1st_sem
+gender_Male +app_over23 +mother_higher + mother_agriculture +
approved_1st_sem +app_special +scholarship,
data = training)
# Predictions in the testing set
lda_preds <- predict(lda_model, newdata = testing)
# Model evaluation
conf_matrix_lda <- confusionMatrix(lda_preds$class, testing$academic_status)
print(conf_matrix_lda)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 221 53 41
## Enrolled 2 4 2
## Graduate 59 100 381
##
## Overall Statistics
##
## Accuracy : 0.7022
## 95% CI : (0.6705, 0.7326)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.479
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.7837 0.025478 0.8986
## Specificity 0.8382 0.994334 0.6378
## Pos Pred Value 0.7016 0.500000 0.7056
## Neg Pred Value 0.8887 0.821053 0.8669
## Prevalence 0.3268 0.181924 0.4913
## Detection Rate 0.2561 0.004635 0.4415
## Detection Prevalence 0.3650 0.009270 0.6257
## Balanced Accuracy 0.8109 0.509906 0.7682
The LDA model performs a bit worse compared to our last model, with an overall accuracy of 70.22% (vs 70.45%). Regarding its performance related to the dropout category, it has a slightly better specificity (83.82% vs 82.27%), but the most important metric in this case is sensitivity in which it performs a bit worse (78.37% vs 79.43%). Our multinom.model2 is the best one up to the moment.
Quadratic Discriminant Analysis (QDA)
Next, we will implement and evaluate QDA model.
# Train QDA model
qda_model <- qda(academic_status~ debtor +mother_no_working +tourism_hospitality
+credits_1st_sem +gender_Male +app_over23 +mother_higher
+mother_agriculture +approved_1st_sem +app_special +scholarship,
data = training)
# Predictions in the testing set
prediction_qda <- predict(qda_model, newdata = testing)$class
# Evaluation
conf_matrix_qda <- confusionMatrix(prediction_qda, testing$academic_status)
print(conf_matrix_qda)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 177 32 35
## Enrolled 74 70 107
## Graduate 31 55 282
##
## Overall Statistics
##
## Accuracy : 0.613
## 95% CI : (0.5796, 0.6456)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : 4.656e-13
##
## Kappa : 0.4001
##
## Mcnemar's Test P-Value : 2.435e-07
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.6277 0.44586 0.6651
## Specificity 0.8847 0.74363 0.8041
## Pos Pred Value 0.7254 0.27888 0.7663
## Neg Pred Value 0.8304 0.85784 0.7131
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2051 0.08111 0.3268
## Detection Prevalence 0.2827 0.29085 0.4264
## Balanced Accuracy 0.7562 0.59474 0.7346
The QDA model performed worse than the previous models in terms of overall accuracy (61.3%) and predict worse in every metric.
Machine Learning
After evaluating these statistical models, we will transition to machine learning techniques to improve prediction accuracy and further refine our understanding of the data.
# Configure control for 10-fold cross validation
ctrl <- trainControl(
method = "repeatedcv",
number = 10,
repeats = 3,
classProbs = TRUE,
summaryFunction = multiClassSummary,
verboseIter = TRUE
)Benchmark model
Our first move will be establishing a Benchmark model. This model predicts the most frequent class, in this case, Graduate, without considering any predictor variables. It serves as a baseline to compare the performance of more advanced models.
# Benchmark: Regla de Mayoría (predice siempre Graduate)
majority_class <- names(which.max(table(training$academic_status)))
benchmark_preds <- factor(rep(majority_class, nrow(testing)),
levels = levels(testing$academic_status))
# Evaluar Benchmark
conf_matrix_benchmark <- confusionMatrix(benchmark_preds, testing$academic_status)
# Imprimir resultados
print(conf_matrix_benchmark)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 0 0 0
## Enrolled 0 0 0
## Graduate 282 157 424
##
## Overall Statistics
##
## Accuracy : 0.4913
## 95% CI : (0.4575, 0.5252)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : 0.5135
##
## Kappa : 0
##
## Mcnemar's Test P-Value : NA
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.0000 0.0000 1.0000
## Specificity 1.0000 1.0000 0.0000
## Pos Pred Value NaN NaN 0.4913
## Neg Pred Value 0.6732 0.8181 NaN
## Prevalence 0.3268 0.1819 0.4913
## Detection Rate 0.0000 0.0000 0.4913
## Detection Prevalence 0.0000 0.0000 1.0000
## Balanced Accuracy 0.5000 0.5000 0.5000
The overall accuracy of the model is 49.94%. This benchmark serves as a minimum reference point: any model we build must exceed it.
k-Nearest Neighbors (k-NN)
Now, we will explore the k-NN algorithm.
set.seed(123)
knnFit <- train(
academic_status ~ .,
data = training,
method = "kknn",
preProc = c('scale', 'center'),
tuneLength = 5,
metric = "Accuracy",
trControl = ctrl
)## + Fold01.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold01.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold01.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold01.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold01.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold01.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold01.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold01.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold01.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold01.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold02.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold02.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold02.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold02.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold02.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold02.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold02.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold02.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold02.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold02.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold03.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold03.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold03.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold03.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold03.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold03.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold03.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold03.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold03.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold03.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold04.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold04.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold04.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold04.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold04.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold04.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold04.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold04.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold04.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold04.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold05.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold05.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold05.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold05.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold05.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold05.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold05.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold05.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold05.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold05.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold06.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold06.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold06.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold06.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold06.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold06.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold06.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold06.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold06.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold06.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold07.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold07.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold07.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold07.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold07.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold07.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold07.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold07.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold07.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold07.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold08.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold08.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold08.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold08.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold08.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold08.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold08.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold08.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold08.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold08.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold09.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold09.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold09.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold09.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold09.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold09.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold09.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold09.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold09.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold09.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold10.Rep1: kmax= 5, distance=2, kernel=optimal
## - Fold10.Rep1: kmax= 5, distance=2, kernel=optimal
## + Fold10.Rep1: kmax= 7, distance=2, kernel=optimal
## - Fold10.Rep1: kmax= 7, distance=2, kernel=optimal
## + Fold10.Rep1: kmax= 9, distance=2, kernel=optimal
## - Fold10.Rep1: kmax= 9, distance=2, kernel=optimal
## + Fold10.Rep1: kmax=11, distance=2, kernel=optimal
## - Fold10.Rep1: kmax=11, distance=2, kernel=optimal
## + Fold10.Rep1: kmax=13, distance=2, kernel=optimal
## - Fold10.Rep1: kmax=13, distance=2, kernel=optimal
## + Fold01.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold01.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold01.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold01.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold01.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold01.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold01.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold01.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold01.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold01.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold02.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold02.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold02.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold02.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold02.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold02.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold02.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold02.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold02.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold02.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold03.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold03.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold03.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold03.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold03.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold03.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold03.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold03.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold03.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold03.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold04.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold04.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold04.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold04.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold04.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold04.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold04.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold04.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold04.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold04.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold05.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold05.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold05.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold05.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold05.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold05.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold05.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold05.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold05.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold05.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold06.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold06.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold06.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold06.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold06.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold06.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold06.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold06.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold06.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold06.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold07.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold07.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold07.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold07.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold07.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold07.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold07.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold07.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold07.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold07.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold08.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold08.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold08.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold08.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold08.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold08.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold08.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold08.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold08.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold08.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold09.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold09.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold09.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold09.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold09.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold09.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold09.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold09.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold09.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold09.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold10.Rep2: kmax= 5, distance=2, kernel=optimal
## - Fold10.Rep2: kmax= 5, distance=2, kernel=optimal
## + Fold10.Rep2: kmax= 7, distance=2, kernel=optimal
## - Fold10.Rep2: kmax= 7, distance=2, kernel=optimal
## + Fold10.Rep2: kmax= 9, distance=2, kernel=optimal
## - Fold10.Rep2: kmax= 9, distance=2, kernel=optimal
## + Fold10.Rep2: kmax=11, distance=2, kernel=optimal
## - Fold10.Rep2: kmax=11, distance=2, kernel=optimal
## + Fold10.Rep2: kmax=13, distance=2, kernel=optimal
## - Fold10.Rep2: kmax=13, distance=2, kernel=optimal
## + Fold01.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold01.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold01.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold01.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold01.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold01.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold01.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold01.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold01.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold01.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold02.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold02.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold02.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold02.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold02.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold02.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold02.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold02.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold02.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold02.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold03.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold03.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold03.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold03.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold03.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold03.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold03.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold03.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold03.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold03.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold04.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold04.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold04.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold04.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold04.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold04.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold04.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold04.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold04.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold04.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold05.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold05.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold05.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold05.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold05.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold05.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold05.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold05.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold05.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold05.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold06.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold06.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold06.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold06.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold06.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold06.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold06.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold06.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold06.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold06.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold07.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold07.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold07.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold07.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold07.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold07.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold07.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold07.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold07.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold07.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold08.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold08.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold08.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold08.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold08.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold08.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold08.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold08.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold08.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold08.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold09.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold09.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold09.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold09.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold09.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold09.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold09.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold09.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold09.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold09.Rep3: kmax=13, distance=2, kernel=optimal
## + Fold10.Rep3: kmax= 5, distance=2, kernel=optimal
## - Fold10.Rep3: kmax= 5, distance=2, kernel=optimal
## + Fold10.Rep3: kmax= 7, distance=2, kernel=optimal
## - Fold10.Rep3: kmax= 7, distance=2, kernel=optimal
## + Fold10.Rep3: kmax= 9, distance=2, kernel=optimal
## - Fold10.Rep3: kmax= 9, distance=2, kernel=optimal
## + Fold10.Rep3: kmax=11, distance=2, kernel=optimal
## - Fold10.Rep3: kmax=11, distance=2, kernel=optimal
## + Fold10.Rep3: kmax=13, distance=2, kernel=optimal
## - Fold10.Rep3: kmax=13, distance=2, kernel=optimal
## Aggregating results
## Selecting tuning parameters
## Fitting kmax = 13, distance = 2, kernel = optimal on full training set
# Predictions
knnProb <- predict(knnFit, testing, type="prob")
prediction_knn <- predict(knnFit, testing)
# Evaluation
conf_matrix_knn <- confusionMatrix(prediction_knn, testing$academic_status)
print(conf_matrix_knn)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 163 33 41
## Enrolled 41 35 31
## Graduate 78 89 352
##
## Overall Statistics
##
## Accuracy : 0.6373
## 95% CI : (0.6042, 0.6695)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.3876
##
## Mcnemar's Test P-Value : 8.755e-09
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.5780 0.22293 0.8302
## Specificity 0.8726 0.89802 0.6196
## Pos Pred Value 0.6878 0.32710 0.6782
## Neg Pred Value 0.8099 0.83862 0.7907
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.1889 0.04056 0.4079
## Detection Prevalence 0.2746 0.12399 0.6014
## Balanced Accuracy 0.7253 0.56047 0.7249
In the results, the k-NN model shows an accuracy of 63.73%, which is better than the benchmark model, but still worse than our last multinomial.
Decision tree
Next, let’s see if the Decision Tree algorithm works better for our dataset.
set.seed(123)
# Train the Decision Tree
control <- rpart.control(minsplit = 8, maxdepth = 12, cp=0.001)
dtFit <- rpart(academic_status ~ ., data = training, method = "class",
control = control)
# Visualization
rpart.plot(dtFit, digits = 3)## Warning: labs do not fit even at cex 0.15, there may be some overplotting
# Prediction
dtPred <- predict(dtFit, testing, type = "class")
# Evaluation
conf_matrix_dt <- confusionMatrix(dtPred, testing$academic_status)
print(conf_matrix_dt)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 194 44 39
## Enrolled 40 38 36
## Graduate 48 75 349
##
## Overall Statistics
##
## Accuracy : 0.6732
## 95% CI : (0.6408, 0.7045)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.4575
##
## Mcnemar's Test P-Value : 0.001973
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.6879 0.24204 0.8231
## Specificity 0.8571 0.89235 0.7198
## Pos Pred Value 0.7004 0.33333 0.7394
## Neg Pred Value 0.8498 0.84112 0.8082
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2248 0.04403 0.4044
## Detection Prevalence 0.3210 0.13210 0.5469
## Balanced Accuracy 0.7725 0.56719 0.7715
In the results, the Decision Tree model shows an accuracy of 67.32%, which is better than the k-NN model but still not better than out winner up to the moment.
Random forest
Now we will try with the Random Forest (RF) model but we will try to reduce its computation time, which was previously too long. To speed up the process, we use parallel processing.
# Activating parallel processing
cl <- makePSOCKcluster(detectCores() - 1)
registerDoParallel(cl)
# Random Forest
rfFit <- train(
academic_status ~ .,
data = training,
method = "rf",
preProc = c('scale', 'center'),
tuneLength = 3,
metric = "Accuracy",
trControl = ctrl,
ntree = 100
)## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,
## : There were missing values in resampled performance measures.
## Aggregating results
## Selecting tuning parameters
## Fitting mtry = 23 on full training set
stopCluster(cl) # Close prallel processes
# Model Evaluation
rfPred <- predict(rfFit, testing)
conf_matrix_rf <- confusionMatrix(rfPred, testing$academic_status)
print(conf_matrix_rf)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 209 46 26
## Enrolled 28 40 18
## Graduate 45 71 380
##
## Overall Statistics
##
## Accuracy : 0.7289
## 95% CI : (0.6979, 0.7583)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.5428
##
## Mcnemar's Test P-Value : 6.461e-09
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.7411 0.25478 0.8962
## Specificity 0.8761 0.93484 0.7358
## Pos Pred Value 0.7438 0.46512 0.7661
## Neg Pred Value 0.8746 0.84942 0.8801
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2422 0.04635 0.4403
## Detection Prevalence 0.3256 0.09965 0.5747
## Balanced Accuracy 0.8086 0.59481 0.8160
This model, along with the multinomial logistic regression model, is one of the best we have achieved so far. It surpasses the multinomial model in terms of accuracy and specificity, but slightly falls behind in terms of sensitivity for the Dropout category.
Gradient Boosting
Finally, we try Gradient Boosting.
# Defining hyperparameters manually to reduce time
param_grid <- expand.grid(
nrounds = c(50, 100),
max_depth = c(3, 5),
eta = c(0.1, 0.2),
gamma = c(0, 0.1),
colsample_bytree = c(0.7),
min_child_weight = c(1),
subsample = c(0.7)
)
# Cross Validation
ctrl <- trainControl(
method = "cv",
number = 3,
classProbs = TRUE,
verboseIter = TRUE,
allowParallel = FALSE
)
# Training XGBoost
gbmFit <- train(
academic_status ~ .,
data = training,
method = "xgbTree",
preProc = c('scale', 'center'),
tuneGrid = param_grid, # En vez de tuneLength
metric = "Accuracy",
trControl = ctrl
)## + Fold1: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold1: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold1: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold2: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold2: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.1, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.1, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.1, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.1, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.2, max_depth=3, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.2, max_depth=3, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.2, max_depth=5, gamma=0.0, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## + Fold3: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## [19:19:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:19:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## - Fold3: eta=0.2, max_depth=5, gamma=0.1, colsample_bytree=0.7, min_child_weight=1, subsample=0.7, nrounds=100
## Aggregating results
## Selecting tuning parameters
## Fitting nrounds = 100, max_depth = 3, eta = 0.1, gamma = 0.1, colsample_bytree = 0.7, min_child_weight = 1, subsample = 0.7 on full training set
# Predictions
gbmProb <- predict(gbmFit, testing, type="prob")
prediction_gbm <- predict(gbmFit, testing)
# Evaluation
conf_matrix_gbm <- confusionMatrix(prediction_gbm, testing$academic_status)
print(conf_matrix_gbm)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 217 45 28
## Enrolled 26 32 20
## Graduate 39 80 376
##
## Overall Statistics
##
## Accuracy : 0.7242
## 95% CI : (0.6931, 0.7538)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.5341
##
## Mcnemar's Test P-Value : 2.596e-09
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.7695 0.20382 0.8868
## Specificity 0.8744 0.93484 0.7289
## Pos Pred Value 0.7483 0.41026 0.7596
## Neg Pred Value 0.8866 0.84076 0.8696
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2514 0.03708 0.4357
## Detection Prevalence 0.3360 0.09038 0.5736
## Balanced Accuracy 0.8219 0.56933 0.8079
We obtained very similar results as with Random Forest. Visualizing the results:
gbm_importance <- varImp(gbmFit, scale = FALSE)
plot(gbm_importance, main = "Feature Importance - Gradient Boosting")From the plot, we get that the most important variables for the Gradient Boosting model are: the number of courses approved and enrolled in the first semester, average grade in the first semester, the number of evaluations to curricular units, age of enrollment and having a scholarship. These variables appear at the top of the list and show the most significant influence on the model’s predictions.
Visual confusion matrix
conf_matrix <- as.data.frame(conf_matrix_gbm$table)
# Visualization
ggplot(conf_matrix, aes(x=Reference, y=Prediction, fill=Freq)) +
geom_tile() +
scale_fill_gradient(low="white", high="red") +
geom_text(aes(label=Freq), vjust=0.5, size=5) +
theme_minimal() +
labs(title="Matriz de Confusión - Gradient Boosting", x="Real Value", y="Prediction")We can visually appreciate how most dropout and graduates cases were correctly identified, being the highest number of false positives in relation to the enrolled category.
Partial Dependence Plots (PDP)
# Crear gráficos de dependencia parcial
partial(gbmFit, pred.var = "scholarship", plot = TRUE, rug = TRUE)The graph shows how the prediction changes as admission_grade varies, holding all other variables constant. The fluctuations in the line indicate that the effect of “admission_grade” is not linear and that the relationship with the prediction changes at different levels of “admission_grade”. In contrast, with “scholarship” we observe a linear relationship.
Ensemble Model
to finish with this first part we are going to create an ensemble model using the three models that performed better for our objectives, that is, the second Multinomial Logistic Regression, Random Forest, and XGBoost, and check if it performs better or not.
# We will use the majority vote of all the models for the final prediction.
ensemble_preds <- data.frame(multinom = test_preds_multinom2,
rf = rfPred,
gbm = prediction_gbm)
# Combine the predictions using a simple majority vote
ensemble_preds_final <- apply(ensemble_preds, 1, function(x)
names(sort(table(x), decreasing = TRUE)[1]))
# Convert final predictions to factor
ensemble_preds_final <- factor(ensemble_preds_final,
levels = levels(testing$academic_status))
# Evaluate the ensemble model
conf_matrix_ensemble <- confusionMatrix(ensemble_preds_final,
testing$academic_status)
print(conf_matrix_ensemble)## Confusion Matrix and Statistics
##
## Reference
## Prediction Dropout Enrolled Graduate
## Dropout 226 55 34
## Enrolled 13 18 9
## Graduate 43 84 381
##
## Overall Statistics
##
## Accuracy : 0.7242
## 95% CI : (0.6931, 0.7538)
## No Information Rate : 0.4913
## P-Value [Acc > NIR] : < 2.2e-16
##
## Kappa : 0.527
##
## Mcnemar's Test P-Value : < 2.2e-16
##
## Statistics by Class:
##
## Class: Dropout Class: Enrolled Class: Graduate
## Sensitivity 0.8014 0.11465 0.8986
## Specificity 0.8468 0.96884 0.7107
## Pos Pred Value 0.7175 0.45000 0.7500
## Neg Pred Value 0.8978 0.83111 0.8789
## Prevalence 0.3268 0.18192 0.4913
## Detection Rate 0.2619 0.02086 0.4415
## Detection Prevalence 0.3650 0.04635 0.5886
## Balanced Accuracy 0.8241 0.54174 0.8046
We obtained the best accuracy so far 72.42%, and a better performance in all metrics for all our classes. This will be our chosen model.
Part 2: Advanced regression
In this second part our research question will be, on the one hand, what are the most important factors influencing students’ academic performance during the first year of university? And, on the other hand, how accurately can we predict their average grades during the first year enrolled od students in Instituto Politécnico de Portalegre using different statistical and machine learning models?
Our target variable will be derived from a combination of first and second semester grades (grade_1st_sem and grade_2nd_sem). The average grades of both semesters will be our target variable.
Consequently, we remove grade_1st_sem and grade_2nd_sem to avoid repeating information. We also exclude the no_eval and approved variables because they contain information that would only be known after the student has completed their courses, which contradicts our goal of building a model that predicts academic performance before the student begins their studies. On the contrary, credits and enrolled variables will be retained because they provide relevant information about the student’s academic workload and engagement at the time of enrollment.
df <- df %>%
dplyr::select( -grade_1st_sem, -grade_2nd_sem,
-approved_1st_sem, -approved_2nd_sem,
-no_eval_1st_sem, -no_eval_2nd_sem)Exploring our new target variable
# Histogram to check distribution
ggplot(df, aes(x = average_grade)) +
geom_histogram(binwidth = 1, fill = "steelblue", color = "black", alpha = 0.7) +
labs(title = "Distribution of Average Grade", x = "Average Grade", y = "Count") +
theme_minimal()
Performing some bivariate analysis
# Scholarship Status vs. Average Grade
ggplot(df, aes(x = factor(academic_status), y = average_grade)) +
geom_boxplot(fill = "lightblue") +
labs(title = "Academic status vs. Average Grade",
x = "Academic status",
y = "Average Grade") +
theme_minimal()Our target variable exhibits significant skewness due to its strong correlation with academic status, where low values primarily reflect dropout students. Since our objective prioritizes academic success forecasting over dropout analysis, recalibrating the dataset to exclude dropout-influenced cases would enhance model precision for the target population.
The box plot reveals several outliers significantly deviating from the main distribution. To address this issue and improve the model’s accuracy, we’ve decided to filter out cases with an average grade below 9. This step helps to focus the analysis on more representative data points and reduces the impact of extreme outliers on our predictions.
ggplot(df, aes(y = average_grade)) +
geom_boxplot(fill = "lightblue", outlier.color = "red", outlier.shape = 16) +
labs(title = "Boxplot de Average Grade", y = "Average Grade") +
theme_minimal()# Removing outliers
df <- df %>% filter(average_grade >= 9)
# Histogram to check distribution
ggplot(df, aes(x = average_grade)) +
geom_histogram(binwidth = 1, fill = "steelblue", color = "black", alpha = 0.7) +
labs(title = "Distribution of Average Grade", x = "Average Grade", y = "Count") +
theme_minimal()More bivariate analysis
# 1. Admission Grade vs. Average Grade
ggplot(df, aes(x = admission_grade, y = average_grade)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", se = FALSE, color = "blue") +
labs(title = "Admission Grade vs. Average Grade",
x = "Admission Grade",
y = "Average Grade") +
theme_minimal()## `geom_smooth()` using formula = 'y ~ x'
# 2. Scholarship Status vs. Average Grade
ggplot(df, aes(x = factor(scholarship, labels = c("No", "Yes")),
y = average_grade)) +
geom_boxplot(fill = "lightblue") +
labs(title = "Scholarship Status vs. Average Grade",
x = "Scholarship Status",
y = "Average Grade") +
theme_minimal()# 3. Age at Enrollment vs. Average Grade
ggplot(df, aes(x = prev_grade, y = average_grade)) +
geom_point(alpha = 0.5) +
geom_smooth(method = "lm", se = FALSE, color = "red") +
labs(title = "Admission Grade vs. Average Grade",
x = "Previous Grade",
y = "Average Grade") +
theme_minimal()## `geom_smooth()` using formula = 'y ~ x'
Students with and without an scholarship don’t seem to have significance
differences in their average grade. However, we observe a positive
relation between our target and the other two variables. The higher the
admission and the previous grade, the highest seem to be their average
grade during the first course.
Correlation analysis and feature selection
We visualize the correlations between numerical variables in the dataset.
# Select only numeric variables
numeric_data <- df[, sapply(df, is.numeric)]
# Compute the correlation matrix
cor_matrix <- cor(numeric_data, use = "complete.obs")
# Install and load the corrplot package
library(corrplot)
# Plot the correlation matrix
corrplot(cor_matrix, method = "color", type = "upper", tl.col = "black", tl.srt = 45)
As we did in the previous part, we identify highly correlated variable
pairs and take decisions to avoid multicollinearity.
# Convert the correlation matrix to a data frame
cor_df <- as.data.frame(as.table(cor_matrix))
# Filter pairs with correlation >= 0.8 (excluding self-correlations)
high_cor_pairs <- subset(cor_df, abs(Freq) > 0.7 & Var1 != Var2)
# Sort by absolute correlation value
high_cor_pairs <- high_cor_pairs[order(-abs(high_cor_pairs$Freq)), ]
# Display the highly correlated pairs
print(high_cor_pairs)## Var1 Var2 Freq
## 575 portuguese international -1.0000000
## 1727 international portuguese -1.0000000
## 604 credits_2nd_sem credits_1st_sem 0.9444184
## 748 credits_1st_sem credits_2nd_sem 0.9444184
## 654 enrolled_2nd_sem enrolled_1st_sem 0.9405905
## 798 enrolled_1st_sem enrolled_2nd_sem 0.9405905
## 602 enrolled_1st_sem credits_1st_sem 0.8795548
## 650 credits_1st_sem enrolled_1st_sem 0.8795548
## 653 credits_2nd_sem enrolled_1st_sem 0.8599589
## 749 enrolled_1st_sem credits_2nd_sem 0.8599589
## 752 enrolled_2nd_sem credits_2nd_sem 0.8155233
## 800 credits_2nd_sem enrolled_2nd_sem 0.8155233
## 605 enrolled_2nd_sem credits_1st_sem 0.7757531
## 797 credits_1st_sem enrolled_2nd_sem 0.7757531
## 1309 secondary_education app_previous_highered -0.7457083
## 1693 app_previous_highered secondary_education -0.7457083
## 1803 mother_basic mother_secondary -0.7119094
## 1899 mother_secondary mother_basic -0.7119094
We average the credits, enrolled y evaluations of the first and second semesters.
df <- df %>%
mutate(
avg_credits = (credits_1st_sem + credits_2nd_sem) / 2,
avg_enrolled = (enrolled_1st_sem + enrolled_2nd_sem) / 2,
avg_evaluations = (evaluations_1st_sem + evaluations_2nd_sem) / 2
) %>%
dplyr::select(-credits_1st_sem, -credits_2nd_sem,
-enrolled_1st_sem, -enrolled_2nd_sem,
-evaluations_1st_sem, -evaluations_2nd_sem)However, avg_enrolled and avg_credits still have a very high correlation so we decide to keep only avg_enrolled because it directly represents the number of courses a student is taking, which might be more intuitive for interpretation. We are also discarding app_previous_highered because its very correlated with prev_secondary.We are not interrested in the academic status of students either.
Interpreting
We divide our dataset into training and testing set.
set.seed(123)
in_train <- createDataPartition(df$average_grade, p = 0.75, list = FALSE)
training <- df[in_train, ]
testing <- df[-in_train, ]
nrow(training)## [1] 1938
## [1] 644
We plot the average_grade distribution in the training set
ggplot(training, aes(x = average_grade)) +
geom_density(fill = "navyblue", alpha = 0.6) +
labs(title = "Distribution of Average Grade", x = "Average Grade",
y = "Density")
We now want to visualize the correlation of the variable we will be
working with and our target variable
num_vars <- training[, sapply(training, is.numeric)]
cor_matrix <- cor(num_vars, use = "complete.obs")
# Extract correlations of average_grade with the rest
corr_average_grade <- sort(cor_matrix["average_grade", ], decreasing = TRUE)
corr_df <- data.frame(Variable = names(corr_average_grade), Correlation = corr_average_grade)
# Graph
ggplot(corr_df, aes(x = reorder(Variable, Correlation), y = Correlation)) +
geom_bar(stat = "identity", fill = "lightblue") +
coord_flip() + # Para mejor visualización
labs(title = "Correlación de Variables con Average Grade",
x = "Predictoras",
y = "Coeficiente de Correlación") +
theme_minimal()
With this graph we identify which variables have the strongest
relationship with our target. These include admission_grade,
health_sciences, prev_grade, avg_evaluations, management_business,
social_sciences_humanities, prev_highe, app_transfer_coursechange,
attendance_Daytime, and mother_higher. This relationships have
theoretical and logical sense
Basing on the previously identified as more relevant variables we will try to build our initial multiple lineal model.
set.seed(123)
lm_model <- lm(average_grade ~ admission_grade + health_sciences + prev_grade +
avg_evaluations + management_business +
social_sciences_humanities + age_enrollment +
app_transfer_coursechange + attendance_Daytime +
mother_higher, data = training)
summary(lm_model)##
## Call:
## lm(formula = average_grade ~ admission_grade + health_sciences +
## prev_grade + avg_evaluations + management_business + social_sciences_humanities +
## age_enrollment + app_transfer_coursechange + attendance_Daytime +
## mother_higher, data = training)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.4101 -0.7205 -0.0772 0.6635 5.0454
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 10.041620 0.312058 32.179 < 2e-16 ***
## admission_grade 0.021016 0.002163 9.715 < 2e-16 ***
## health_sciences 0.581277 0.058224 9.984 < 2e-16 ***
## prev_grade 0.005077 0.002310 2.198 0.028064 *
## avg_evaluations -0.114360 0.008776 -13.031 < 2e-16 ***
## management_business -0.637748 0.079052 -8.067 1.25e-15 ***
## social_sciences_humanities -0.259947 0.068579 -3.790 0.000155 ***
## age_enrollment 0.011995 0.004255 2.819 0.004865 **
## app_transfer_coursechange 0.223885 0.090550 2.472 0.013503 *
## attendance_Daytime 0.069819 0.100645 0.694 0.487947
## mother_higher 0.206329 0.069205 2.981 0.002905 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 1.016 on 1927 degrees of freedom
## Multiple R-squared: 0.2699, Adjusted R-squared: 0.2661
## F-statistic: 71.24 on 10 and 1927 DF, p-value: < 2.2e-16
The multiple linear regression model explains 26.99% of the variance in average_grade (R² = 0.2699), with an adjusted R² of 0.2661, indicating that while some predictors are significant, the model does not capture all the variation in student performance. Most of our selected variables are highly significant.
We can highlight some observations. Higher admission grades lead to better academic performance. Moreover, students in health sciences, those with higher previous education, and those with higher-educated mothers tend to perform better. On the contrary, more evaluations correlate with lower grades. The same way, students in fields like management and business or social sciences and humanities are associated with lower average grades. There is no strong evidence that daytime attendance impacts performance.
Evaluating normality of the residuals
Our residuals are normally distributed.
Let’s try another model. The previous one focused on individual characteristics, so now we will include variables related to the economical context and their academic specialization.
set.seed(123)
lm_model2 <- lm(average_grade ~ avg_evaluations+ avg_enrolled + arts_media_comm
+ agriculture_env + health_sciences + engineering_tech +
app_transfer_coursechange + app_transfer_coursechange +
app_regular + gdp + unemployment_rate + age_enrollment +
admission_grade + prev_grade + mother_higher, data = training)
summary(lm_model2)##
## Call:
## lm(formula = average_grade ~ avg_evaluations + avg_enrolled +
## arts_media_comm + agriculture_env + health_sciences + engineering_tech +
## app_transfer_coursechange + app_transfer_coursechange + app_regular +
## gdp + unemployment_rate + age_enrollment + admission_grade +
## prev_grade + mother_higher, data = training)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.6557 -0.6675 -0.0648 0.6041 4.4268
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 8.802266 0.329063 26.750 < 2e-16 ***
## avg_evaluations -0.195966 0.011073 -17.698 < 2e-16 ***
## avg_enrolled 0.202425 0.016436 12.316 < 2e-16 ***
## arts_media_comm 0.309529 0.060067 5.153 2.82e-07 ***
## agriculture_env 0.661855 0.098120 6.745 2.01e-11 ***
## health_sciences 0.857844 0.057875 14.822 < 2e-16 ***
## engineering_tech 0.440848 0.142307 3.098 0.00198 **
## app_transfer_coursechange 0.072234 0.100404 0.719 0.47196
## app_regular 0.214556 0.071206 3.013 0.00262 **
## gdp 0.059126 0.010667 5.543 3.39e-08 ***
## unemployment_rate 0.042802 0.009084 4.712 2.63e-06 ***
## age_enrollment 0.013823 0.004390 3.149 0.00166 **
## admission_grade 0.019942 0.002092 9.533 < 2e-16 ***
## prev_grade 0.003612 0.002209 1.635 0.10221
## mother_higher 0.172884 0.066421 2.603 0.00932 **
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9647 on 1923 degrees of freedom
## Multiple R-squared: 0.3426, Adjusted R-squared: 0.3378
## F-statistic: 71.59 on 14 and 1923 DF, p-value: < 2.2e-16
This model performs better, having an adjusted R-squared of 0.3378 and being most of the variables included significant.
Next, we will aplly some methods for improving model accuracy by adjusting variable selection efficiently. This procedures will help us to identify the most relevant variables while discarding the less informative ones.
##
##
## Stepwise Summary
## -----------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -----------------------------------------------------------------------------------
## 0 Base Model 6162.388 6173.527 661.680 0.00000 0.00000
## 1 admission_grade 5965.750 5982.459 464.739 0.09742 0.09695
## 2 health_sciences 5810.176 5832.453 309.064 0.16790 0.16704
## 3 avg_evaluations 5661.630 5689.477 160.670 0.23009 0.22890
## 4 avg_enrolled 5487.491 5520.907 -12.898 0.29698 0.29553
## 5 agriculture_env 5453.910 5492.896 -46.381 0.30977 0.30798
## 6 arts_media_comm 5431.458 5476.013 -68.741 0.31843 0.31631
## 7 gdp 5414.281 5464.405 -85.819 0.32514 0.32269
## 8 unemployment_rate 5395.447 5451.141 -104.498 0.33235 0.32958
## 9 engineering_tech 5388.516 5449.779 -111.353 0.33542 0.33232
## 10 mother_higher 5383.888 5450.721 -115.912 0.33769 0.33425
## 11 app_regular 5383.493 5455.895 -116.276 0.33851 0.33473
## 12 age_enrollment 5376.754 5454.726 -122.895 0.34149 0.33738
## 13 prev_grade 5375.915 5459.456 -123.680 0.34245 0.33801
## -----------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## ----------------------------------------------------------------
## R 0.585 RMSE 0.961
## R-Squared 0.342 MSE 0.924
## Adj. R-Squared 0.338 Coef. Var 7.541
## Pred R-Squared 0.331 AIC 5375.915
## MAE 0.758 SBC 5459.456
## ----------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## ----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## ----------------------------------------------------------------------
## Regression 932.254 13 71.712 77.078 0.0000
## Residual 1790.061 1924 0.930
## Total 2722.315 1937
## ----------------------------------------------------------------------
##
## Parameter Estimates
## -----------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## -----------------------------------------------------------------------------------------------
## (Intercept) 8.849 0.323 27.429 0.000 8.216 9.481
## admission_grade 0.020 0.002 0.233 9.584 0.000 0.016 0.024
## health_sciences 0.853 0.057 0.323 14.849 0.000 0.740 0.965
## avg_evaluations -0.196 0.011 -0.478 -17.689 0.000 -0.218 -0.174
## avg_enrolled 0.206 0.016 0.342 13.109 0.000 0.175 0.237
## agriculture_env 0.651 0.097 0.139 6.718 0.000 0.461 0.840
## arts_media_comm 0.310 0.060 0.110 5.169 0.000 0.193 0.428
## gdp 0.058 0.011 0.110 5.497 0.000 0.037 0.079
## unemployment_rate 0.042 0.009 0.093 4.662 0.000 0.024 0.060
## engineering_tech 0.439 0.142 0.059 3.087 0.002 0.160 0.718
## mother_higher 0.176 0.066 0.050 2.652 0.008 0.046 0.306
## app_regular 0.192 0.064 0.075 3.012 0.003 0.067 0.316
## age_enrollment 0.013 0.004 0.075 3.069 0.002 0.005 0.022
## prev_grade 0.004 0.002 0.041 1.679 0.093 -0.001 0.008
## -----------------------------------------------------------------------------------------------
##
##
## Stepwise Summary
## -----------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -----------------------------------------------------------------------------------
## 0 Base Model 6162.388 6173.527 661.680 0.00000 0.00000
## 1 admission_grade 5965.750 5982.459 464.739 0.09742 0.09695
## 2 health_sciences 5810.176 5832.453 309.064 0.16790 0.16704
## 3 avg_evaluations 5661.630 5689.477 160.670 0.23009 0.22890
## 4 avg_enrolled 5487.491 5520.907 -12.898 0.29698 0.29553
## 5 agriculture_env 5453.910 5492.896 -46.381 0.30977 0.30798
## 6 arts_media_comm 5431.458 5476.013 -68.741 0.31843 0.31631
## 7 gdp 5414.281 5464.405 -85.819 0.32514 0.32269
## 8 unemployment_rate 5395.447 5451.141 -104.498 0.33235 0.32958
## 9 engineering_tech 5388.516 5449.779 -111.353 0.33542 0.33232
## 10 mother_higher 5383.888 5450.721 -115.912 0.33769 0.33425
## 11 app_regular 5383.493 5455.895 -116.276 0.33851 0.33473
## 12 age_enrollment 5376.754 5454.726 -122.895 0.34149 0.33738
## 13 prev_grade 5375.915 5459.456 -123.680 0.34245 0.33801
## -----------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## ----------------------------------------------------------------
## R 0.585 RMSE 0.961
## R-Squared 0.342 MSE 0.924
## Adj. R-Squared 0.338 Coef. Var 7.541
## Pred R-Squared 0.331 AIC 5375.915
## MAE 0.758 SBC 5459.456
## ----------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## ----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## ----------------------------------------------------------------------
## Regression 932.254 13 71.712 77.078 0.0000
## Residual 1790.061 1924 0.930
## Total 2722.315 1937
## ----------------------------------------------------------------------
##
## Parameter Estimates
## -----------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## -----------------------------------------------------------------------------------------------
## (Intercept) 8.849 0.323 27.429 0.000 8.216 9.481
## admission_grade 0.020 0.002 0.233 9.584 0.000 0.016 0.024
## health_sciences 0.853 0.057 0.323 14.849 0.000 0.740 0.965
## avg_evaluations -0.196 0.011 -0.478 -17.689 0.000 -0.218 -0.174
## avg_enrolled 0.206 0.016 0.342 13.109 0.000 0.175 0.237
## agriculture_env 0.651 0.097 0.139 6.718 0.000 0.461 0.840
## arts_media_comm 0.310 0.060 0.110 5.169 0.000 0.193 0.428
## gdp 0.058 0.011 0.110 5.497 0.000 0.037 0.079
## unemployment_rate 0.042 0.009 0.093 4.662 0.000 0.024 0.060
## engineering_tech 0.439 0.142 0.059 3.087 0.002 0.160 0.718
## mother_higher 0.176 0.066 0.050 2.652 0.008 0.046 0.306
## app_regular 0.192 0.064 0.075 3.012 0.003 0.067 0.316
## age_enrollment 0.013 0.004 0.075 3.069 0.002 0.005 0.022
## prev_grade 0.004 0.002 0.041 1.679 0.093 -0.001 0.008
## -----------------------------------------------------------------------------------------------
##
##
## Stepwise Summary
## ---------------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## ---------------------------------------------------------------------------------------
## 0 Base Model 6162.388 6173.527 661.680 0.00000 0.00000
## 1 admission_grade (+) 5965.750 5982.459 464.739 0.09742 0.09695
## 2 health_sciences (+) 5810.176 5832.453 309.064 0.16790 0.16704
## 3 avg_evaluations (+) 5661.630 5689.477 160.670 0.23009 0.22890
## 4 avg_enrolled (+) 5487.491 5520.907 -12.898 0.29698 0.29553
## 5 agriculture_env (+) 5453.910 5492.896 -46.381 0.30977 0.30798
## 6 arts_media_comm (+) 5431.458 5476.013 -68.741 0.31843 0.31631
## 7 gdp (+) 5414.281 5464.405 -85.819 0.32514 0.32269
## 8 unemployment_rate (+) 5395.447 5451.141 -104.498 0.33235 0.32958
## 9 engineering_tech (+) 5388.516 5449.779 -111.353 0.33542 0.33232
## 10 mother_higher (+) 5383.888 5450.721 -115.912 0.33769 0.33425
## 11 app_regular (+) 5383.493 5455.895 -116.276 0.33851 0.33473
## 12 age_enrollment (+) 5376.754 5454.726 -122.895 0.34149 0.33738
## 13 prev_grade (+) 5375.915 5459.456 -123.680 0.34245 0.33801
## ---------------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## ----------------------------------------------------------------
## R 0.585 RMSE 0.961
## R-Squared 0.342 MSE 0.924
## Adj. R-Squared 0.338 Coef. Var 7.541
## Pred R-Squared 0.331 AIC 5375.915
## MAE 0.758 SBC 5459.456
## ----------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## ----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## ----------------------------------------------------------------------
## Regression 932.254 13 71.712 77.078 0.0000
## Residual 1790.061 1924 0.930
## Total 2722.315 1937
## ----------------------------------------------------------------------
##
## Parameter Estimates
## -----------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## -----------------------------------------------------------------------------------------------
## (Intercept) 8.849 0.323 27.429 0.000 8.216 9.481
## admission_grade 0.020 0.002 0.233 9.584 0.000 0.016 0.024
## health_sciences 0.853 0.057 0.323 14.849 0.000 0.740 0.965
## avg_evaluations -0.196 0.011 -0.478 -17.689 0.000 -0.218 -0.174
## avg_enrolled 0.206 0.016 0.342 13.109 0.000 0.175 0.237
## agriculture_env 0.651 0.097 0.139 6.718 0.000 0.461 0.840
## arts_media_comm 0.310 0.060 0.110 5.169 0.000 0.193 0.428
## gdp 0.058 0.011 0.110 5.497 0.000 0.037 0.079
## unemployment_rate 0.042 0.009 0.093 4.662 0.000 0.024 0.060
## engineering_tech 0.439 0.142 0.059 3.087 0.002 0.160 0.718
## mother_higher 0.176 0.066 0.050 2.652 0.008 0.046 0.306
## app_regular 0.192 0.064 0.075 3.012 0.003 0.067 0.316
## age_enrollment 0.013 0.004 0.075 3.069 0.002 0.005 0.022
## prev_grade 0.004 0.002 0.041 1.679 0.093 -0.001 0.008
## -----------------------------------------------------------------------------------------------
The three methods select the same variables. This pattern is indicative that these variables have a stronger impact on the prediction of the target variable average_grade, which aligns with our intuition about the factors affecting academic performance.
set.seed(123)
linFit <- lm(average_grade ~ admission_grade + health_sciences+ avg_evaluations
+ avg_enrolled + agriculture_env + arts_media_comm + gdp +
unemployment_rate, data=training)
summary(linFit)##
## Call:
## lm(formula = average_grade ~ admission_grade + health_sciences +
## avg_evaluations + avg_enrolled + agriculture_env + arts_media_comm +
## gdp + unemployment_rate, data = training)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.3952 -0.6845 -0.0470 0.5920 4.4771
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 9.636876 0.241615 39.885 < 2e-16 ***
## admission_grade 0.021448 0.001610 13.325 < 2e-16 ***
## health_sciences 0.819471 0.055252 14.832 < 2e-16 ***
## avg_evaluations -0.197482 0.010817 -18.257 < 2e-16 ***
## avg_enrolled 0.202227 0.015532 13.020 < 2e-16 ***
## agriculture_env 0.608813 0.095141 6.399 1.96e-10 ***
## arts_media_comm 0.280682 0.058612 4.789 1.81e-06 ***
## gdp 0.059479 0.010530 5.649 1.86e-08 ***
## unemployment_rate 0.041172 0.009017 4.566 5.28e-06 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9707 on 1929 degrees of freedom
## Multiple R-squared: 0.3324, Adjusted R-squared: 0.3296
## F-statistic: 120 on 8 and 1929 DF, p-value: < 2.2e-16
This models works a bit worse but give us valuable information about the fact that these seem to be the most explanatory variables when trying to understand the differences in the average grades among students.
Prediction
The second goal was trying to predict. Consequently, we are going to evaluate our three linear regression models on the testing set to compare their performance.
# Creating a DataFrame to store predictions and real values
test_results <- data.frame(
observed = testing$average_grade)
# Storing predictions of our models
test_results$lm1 <- predict(lm_model, newdata = testing)
test_results$lm2 <- predict(lm_model2, newdata = testing)
test_results$linFit <- predict(linFit, newdata = testing)
# Evaluating each model in the testing set
metrics_lm1 <- postResample(pred = test_results$lm1, obs = test_results$observed)
metrics_lm2 <- postResample(pred = test_results$lm2, obs = test_results$observed)
metrics_linFit <- postResample(pred = test_results$linFit, obs = test_results$observed)
# Results
cat("\n🔹 lm_model:\n")##
## 🔹 lm_model:
## RMSE Rsquared MAE
## 1.0909497 0.2129775 0.8408353
##
## 🔹 lm_model2:
## RMSE Rsquared MAE
## 1.0630537 0.2560975 0.8077537
##
## 🔹 linFit:
## RMSE Rsquared MAE
## 1.0766453 0.2384323 0.8153802
lm_model2 appears to be the best model based on its lower RMSE and MAE values, as well as the higher R-squared compared to the other models. However, the improvements are relatively small between the models, suggesting that they all perform similarly but with slight differences in accuracy.
Statistical learning tools
We will first explore Statistical Learning Tools to establish a solid foundation. We will perform a variety of regression techniques with which we aim to refine our model selection process and improve the accuracy of our predictions.
Baseline Model in multiple linear regression
Next, we will create a Benchmark model which will be used as a reference for the rest of the models
## [1] 12.79164
# This is equivalent to
benchFit <- lm(average_grade ~ 1, data=training)
predictions <- predict(benchFit, newdata=testing)
RMSE_benchmark <- sqrt(mean((predictions - testing$average_grade)^2))
cat("Benchmark RMSE:", round(RMSE_benchmark, 4), "\n")## Benchmark RMSE: 1.2293
Our baseline model predicts the same value, which is the mean of the target, for all observations, that is, 12.79164. Its RMSE is 1.229268, indicating the average deviation of the predictions from the actual values in the test set.
Foward regression
Let’s try to enhance lm_model2 with forward regression. We will store it as model_formula.
unregister_dopar <- function() {
env <- foreach:::.foreachGlobals
rm(list=ls(name=env), pos=env)
}
unregister_dopar()
model_formula <- average_grade ~ avg_evaluations + avg_enrolled +
arts_media_comm + agriculture_env + health_sciences + engineering_tech +
app_transfer_coursechange + app_transfer_coursechange + app_regular +
gdp + unemployment_rate + age_enrollment + admission_grade +
prev_grade + mother_higher
set.seed(123)
forward_model <- train(
model_formula,
data = training,
method = "leapForward", # Forward Selection en `caret`
preProc = c("scale", "center"), # Normaliza los datos
tuneGrid = expand.grid(nvmax = 4:15), # Busca entre 4 y 15 variables
trControl = ctrl
)
# Show results
print(forward_model)## Linear Regression with Forward Selection
##
## 1938 samples
## 14 predictor
##
## Pre-processing: scaled (14), centered (14)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## nvmax RMSE Rsquared MAE
## 4 0.9960504 0.2953438 0.7859543
## 5 0.9874094 0.3067943 0.7808447
## 6 0.9858782 0.3089667 0.7811261
## 7 0.9824295 0.3136203 0.7781103
## 8 0.9754288 0.3238876 0.7717729
## 9 0.9715475 0.3294321 0.7687518
## 10 0.9732318 0.3275429 0.7699175
## 11 0.9732137 0.3275293 0.7689110
## 12 0.9709911 0.3305934 0.7669721
## 13 0.9682910 0.3344270 0.7643495
## 14 0.9689263 0.3335756 0.7649670
## 15 0.9689263 0.3335756 0.7649670
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was nvmax = 13.
As the number of predictors increased, the RMSE slightly decreased,
showing improvement in the model’s predictive accuracy. The model with
14 predictors had the smallest RMSE of 0.9737, the highest R-squared
value of 0.3270, and an MAE of 0.7694, suggesting that adding more
variables led to a better fit to the data. The results suggest that
incorporating all 14 predictors provided the best balance between model
accuracy and complexity.
Backward regression
set.seed(123)
backward_model <- train(
model_formula,
data = training,
method = "leapBackward", # Backward Selection en `caret`
preProc = c("scale", "center"), # Normaliza los datos
tuneGrid = expand.grid(nvmax = 4:15), # Busca entre 4 y 15 variables
trControl = ctrl
)
# Mostrar resultados
print(backward_model)## Linear Regression with Backwards Selection
##
## 1938 samples
## 14 predictor
##
## Pre-processing: scaled (14), centered (14)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## nvmax RMSE Rsquared MAE
## 4 0.9960504 0.2953438 0.7859543
## 5 0.9885919 0.3052247 0.7813013
## 6 0.9865024 0.3078465 0.7814397
## 7 0.9807314 0.3159736 0.7751869
## 8 0.9725020 0.3280013 0.7674992
## 9 0.9734748 0.3270816 0.7684034
## 10 0.9742894 0.3261859 0.7709900
## 11 0.9733509 0.3273129 0.7687690
## 12 0.9689146 0.3334966 0.7649814
## 13 0.9682910 0.3344270 0.7643495
## 14 0.9689263 0.3335756 0.7649670
## 15 0.9689263 0.3335756 0.7649670
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was nvmax = 13.
Similarly to the forward results, using backward regression we reach a
minimum RMSE value of 0.9692 with 14 predictors. The R-squared value
continued to improve with more predictors, reaching 0.3327 at the final
model, while MAE slightly decreased.
Stepwise regression
set.seed(123)
stepwise_model <- train(
model_formula,
data = training,
method = "leapSeq", # Stepwise Selection en `caret`
preProc = c("scale", "center"), # Normaliza los datos
tuneGrid = expand.grid(nvmax = 4:15), # Busca entre 4 y 15 variables
trControl = ctrl
)
# Mostrar resultados
print(stepwise_model)## Linear Regression with Stepwise Selection
##
## 1938 samples
## 14 predictor
##
## Pre-processing: scaled (14), centered (14)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## nvmax RMSE Rsquared MAE
## 4 1.0035029 0.2836483 0.7913892
## 5 0.9874094 0.3067943 0.7808447
## 6 1.0073687 0.2777834 0.7984871
## 7 0.9917327 0.3004738 0.7809033
## 8 0.9955014 0.2949272 0.7818203
## 9 0.9937667 0.2978710 0.7823203
## 10 0.9950276 0.2962215 0.7842422
## 11 0.9751242 0.3237965 0.7704189
## 12 0.9699659 0.3321967 0.7657664
## 13 0.9687398 0.3337258 0.7633582
## 14 0.9689263 0.3335756 0.7649670
## 15 0.9689263 0.3335756 0.7649670
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was nvmax = 13.
With stepwise regression the optimal model was obtained with 12
predictors, which yielded the lowest RMSE value of 0.9686. The model
also showed a steady improvement in R-squared, reaching 0.3333.
Now we will evaluate and compare the performance of three different variable selection methods on our linear regression model. For each model, we make predictions using the testing set and then assess their accuracy.
set.seed(123)
# Creating a DataFrame to store observations and predictions
test_results <- data.frame(observed = testing$average_grade)
# Making predictions
test_results$fw <- predict(forward_model, newdata = testing)
test_results$bw <- predict(backward_model, newdata = testing)
test_results$sw <- predict(stepwise_model, newdata = testing)
# Evaluating each model
metrics_fw <- postResample(pred = test_results$fw, obs = test_results$observed)
metrics_bw <- postResample(pred = test_results$bw, obs = test_results$observed)
metrics_sw <- postResample(pred = test_results$sw, obs = test_results$observed)
# Show metrics
cat("\n🔹 Forward Selection Metrics:\n")##
## 🔹 Forward Selection Metrics:
## RMSE Rsquared MAE
## 1.0648513 0.2538339 0.8093230
##
## 🔹 Backward Selection Metrics:
## RMSE Rsquared MAE
## 1.0648513 0.2538339 0.8093230
##
## 🔹 Stepwise Selection Metrics:
## RMSE Rsquared MAE
## 1.0648513 0.2538339 0.8093230
Forward and Backward selection techniques provide very similar and slightly better performance compared to Stepwise selection, making them more favorable in this case.
Lasso regression
We proceed to apply regularization method techniques, starting with Lasso.
set.seed(123)
# Defining grid
lasso_grid <- expand.grid(fraction = seq(0.01, 1, length = 100))
# Entrenar el modelo Lasso
set.seed(123)
lasso_tune <- train(model_formula, data = training,
method = "lasso",
preProc = c("scale", "center"),
tuneGrid = lasso_grid,
trControl = ctrl)
# Graficar el efecto de lambda en el RMSE
plot(lasso_tune)## fraction
## 98 0.98
# Crear DataFrame para almacenar predicciones y valores reales
test_results <- data.frame(
observed = testing$average_grade, # Valores reales
lasso = predict(lasso_tune, newdata = testing) # Predicciones del modelo
)
# Evaluar el rendimiento del modelo
metrics_lasso <- postResample(pred = test_results$lasso, obs = test_results$observed)
print(metrics_lasso)## RMSE Rsquared MAE
## 1.0634924 0.2550914 0.8084476
The Lasso model with the best lambda value (0.95) improves only very slightly the RMSE and Rsquared compared to models without regularization. In general we are not having good results
Ridge regression
ridge_grid <- expand.grid(lambda = seq(0, .1, length = 100))
# Training Ridge
set.seed(123)
ridge_tune <- train(model_formula, data = training,
method = "ridge",
preProc = c("scale", "center"),
tuneGrid = ridge_grid,
trControl = ctrl)
# Graph
plot(ridge_tune)## lambda
## 11 0.01010101
# DataFrame to store predictions and real values
test_results <- data.frame(
observed = testing$average_grade,
ridge = predict(ridge_tune, newdata = testing))
# Performance metrics
metrics_ridge <- postResample(pred = test_results$ridge, obs = test_results$observed)
print(metrics_ridge)## RMSE Rsquared MAE
## 1.0628071 0.2559616 0.8081206
We obtain almost the same as with Lasso.
Elastic Net regression
elastic_grid <- expand.grid(alpha = seq(0, 1, 0.1), lambda = seq(0, .1, 0.01))
set.seed(123)
elastic_tune <- train(model_formula, data = training,
method = "glmnet",
preProc = c("scale", "center"),
tuneGrid = elastic_grid,
trControl = ctrl)
plot(elastic_tune)## alpha lambda
## 13 0.1 0.01
test_results <- data.frame(
observed = testing$average_grade,
elastic = predict(elastic_tune, newdata = testing))
metrics_elastic <- postResample(pred = test_results$elastic,
obs = test_results$observed)
print(metrics_elastic)## RMSE Rsquared MAE
## 1.0628722 0.2553720 0.8085837
We also have almost the same performance with Elastic Net.
In general, all the models tested have performed moderately, with results indicating a fair amount of prediction error and limited explanatory power, as shown by relatively low R-squared values and RMSE. This could be due to several factors, such as the complexity of the data or the limitations when using traditional statistical models. Moreover, the linear nature of the models might not be capturing all the non-linear relationships within the data.
Machine Learning
We will now transition to machine learning techniques, which are better equipped to handle complex relationships and interactions in the data. These methods may offer improvements in prediction accuracy.
k-Nearest Neighbors (k-NN)
set.seed(123)
# Defining KNN
knn_tune <- train(average_grade ~ .,
data = training,
method = "kknn",
preProc = c('scale', 'center'),
tuneGrid = data.frame(kmax = c(5, 10, 15, 20),
distance = 2, kernel = 'optimal'),
trControl = ctrl)
# Mostrar los resultados del tuning
print(knn_tune)## k-Nearest Neighbors
##
## 1938 samples
## 41 predictor
##
## Pre-processing: scaled (41), centered (41)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## kmax RMSE Rsquared MAE
## 5 1.160551 0.1362891 0.9056258
## 10 1.101026 0.1643669 0.8647512
## 15 1.083114 0.1764108 0.8528798
## 20 1.074406 0.1846833 0.8474628
##
## Tuning parameter 'distance' was held constant at a value of 2
## Tuning
## parameter 'kernel' was held constant at a value of optimal
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were kmax = 20, distance = 2 and kernel
## = optimal.
# Hacer predicciones en el conjunto de prueba
test_results$knn <- predict(knn_tune, testing)
# Evaluar el modelo KNN
knn_metrics <- postResample(pred = test_results$knn, obs = testing$average_grade)
print(knn_metrics)## RMSE Rsquared MAE
## 1.1490845 0.1353695 0.8898548
The optimal model was selected with kmax = 20, which means the model considers the average of the 20 nearest neighbors for each prediction. The metrics obtained were the worst up to the moment.
Random Forest
set.seed(123)
# Define the model
rf_tune <- train(average_grade ~ .,
data = training,
method = "rf",
preProc = c('scale', 'center'),
trControl = ctrl,
ntree = 100,
tuneGrid = expand.grid(mtry = c(1, 3, 5, 7)),
importance = TRUE)
# Results
print(rf_tune)## Random Forest
##
## 1938 samples
## 41 predictor
##
## Pre-processing: scaled (41), centered (41)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## mtry RMSE Rsquared MAE
## 1 1.0929944 0.2905785 0.8726091
## 3 0.9841443 0.3437907 0.7777923
## 5 0.9609172 0.3566022 0.7593463
## 7 0.9538195 0.3607544 0.7525339
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was mtry = 7.
# Predictions on testing set
test_results$rf <- predict(rf_tune, testing)
# Evaluation
rf_metrics <- postResample(pred = test_results$rf, obs = testing$average_grade)
print(rf_metrics)## RMSE Rsquared MAE
## 1.0098770 0.3317068 0.7651697
Random Forest performed better than earlier models, with lower RMSE and higher R-squared. The best model was selected with mtry = 7, meaning the model will consider 7 features at each split of the decision tree. This one appears to be one of the better-performing models so far (RMSE = 1.0133870 and Rsquared = 0.3255387), although it doesn’t predict very well either.
Decision Tree
set.seed(123)
# Defining model
dt_tune <- train(average_grade ~ .,
data = training,
method = "rpart",
preProc = c('scale', 'center'),
tuneGrid = expand.grid(cp = seq(0.001, 0.05, by = 0.005)),
trControl = ctrl)
# Results
print(dt_tune)## CART
##
## 1938 samples
## 41 predictor
##
## Pre-processing: scaled (41), centered (41)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## cp RMSE Rsquared MAE
## 0.001 1.083498 0.2314922 0.8509588
## 0.006 1.054242 0.2241665 0.8264577
## 0.011 1.049044 0.2214021 0.8276036
## 0.016 1.058506 0.2047494 0.8419989
## 0.021 1.069078 0.1890817 0.8538955
## 0.026 1.078695 0.1752971 0.8614020
## 0.031 1.091076 0.1565998 0.8710403
## 0.036 1.091076 0.1565998 0.8710403
## 0.041 1.106801 0.1314176 0.8843474
## 0.046 1.122416 0.1066643 0.8969738
##
## RMSE was used to select the optimal model using the smallest value.
## The final value used for the model was cp = 0.011.
# Predictions
test_results$dt <- predict(dt_tune, testing)
# Evaluation
dt_metrics <- postResample(pred = test_results$dt, obs = testing$average_grade)
print(dt_metrics)## RMSE Rsquared MAE
## 1.1218153 0.1720713 0.8738138
The Decision Tree performs worse.
Gradient Boosting
set.seed(123)
# Definir modelo Gradient Boosting
xgb_tune <- train(average_grade ~ .,
data = training,
method = "xgbTree",
preProc = c('scale', 'center'),
trControl = ctrl,
tuneGrid = expand.grid(
nrounds = c(5, 100), # Número de árboles PONIA 500, 1000
max_depth = c(3, 5, 7), # Profundidad máxima del árbol
eta = c(0.01, 0.1, 0.3), # Tasa de aprendizaje
gamma = c(0, 1, 3), # Reducción mínima de pérdida
colsample_bytree = c(0.8, 1), # Fracción de características usadas
min_child_weight = c(1, 3), # Peso mínimo de la instancia hija
subsample = c(0.8, 1) # Tasa de muestreo
))## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:20:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:21:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:22:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:26] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:27] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:28] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:29] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:30] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:31] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:32] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:33] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:34] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:35] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:36] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:37] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:38] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:39] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:40] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:41] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:42] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:43] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:44] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:45] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:46] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:47] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:48] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:49] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:50] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:51] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:52] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:53] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:54] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:55] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:56] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:57] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:58] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:23:59] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:00] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:01] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:02] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:03] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:04] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:05] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:06] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:07] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:08] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:09] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:10] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:11] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:12] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:13] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:14] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:15] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:16] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:17] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:18] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:19] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:20] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:21] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:22] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:23] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:24] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## [19:24:25] WARNING: src/c_api/c_api.cc:935: `ntree_limit` is deprecated, use `iteration_range` instead.
## Warning in nominalTrainWorkflow(x = x, y = y, wts = weights, info = trainInfo,
## : There were missing values in resampled performance measures.
## eXtreme Gradient Boosting
##
## 1938 samples
## 41 predictor
##
## Pre-processing: scaled (41), centered (41)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## eta max_depth gamma colsample_bytree min_child_weight subsample
## 0.01 3 0 0.8 1 0.8
## 0.01 3 0 0.8 1 0.8
## 0.01 3 0 0.8 1 1.0
## 0.01 3 0 0.8 1 1.0
## 0.01 3 0 0.8 3 0.8
## 0.01 3 0 0.8 3 0.8
## 0.01 3 0 0.8 3 1.0
## 0.01 3 0 0.8 3 1.0
## 0.01 3 0 1.0 1 0.8
## 0.01 3 0 1.0 1 0.8
## 0.01 3 0 1.0 1 1.0
## 0.01 3 0 1.0 1 1.0
## 0.01 3 0 1.0 3 0.8
## 0.01 3 0 1.0 3 0.8
## 0.01 3 0 1.0 3 1.0
## 0.01 3 0 1.0 3 1.0
## 0.01 3 1 0.8 1 0.8
## 0.01 3 1 0.8 1 0.8
## 0.01 3 1 0.8 1 1.0
## 0.01 3 1 0.8 1 1.0
## 0.01 3 1 0.8 3 0.8
## 0.01 3 1 0.8 3 0.8
## 0.01 3 1 0.8 3 1.0
## 0.01 3 1 0.8 3 1.0
## 0.01 3 1 1.0 1 0.8
## 0.01 3 1 1.0 1 0.8
## 0.01 3 1 1.0 1 1.0
## 0.01 3 1 1.0 1 1.0
## 0.01 3 1 1.0 3 0.8
## 0.01 3 1 1.0 3 0.8
## 0.01 3 1 1.0 3 1.0
## 0.01 3 1 1.0 3 1.0
## 0.01 3 3 0.8 1 0.8
## 0.01 3 3 0.8 1 0.8
## 0.01 3 3 0.8 1 1.0
## 0.01 3 3 0.8 1 1.0
## 0.01 3 3 0.8 3 0.8
## 0.01 3 3 0.8 3 0.8
## 0.01 3 3 0.8 3 1.0
## 0.01 3 3 0.8 3 1.0
## 0.01 3 3 1.0 1 0.8
## 0.01 3 3 1.0 1 0.8
## 0.01 3 3 1.0 1 1.0
## 0.01 3 3 1.0 1 1.0
## 0.01 3 3 1.0 3 0.8
## 0.01 3 3 1.0 3 0.8
## 0.01 3 3 1.0 3 1.0
## 0.01 3 3 1.0 3 1.0
## 0.01 5 0 0.8 1 0.8
## 0.01 5 0 0.8 1 0.8
## 0.01 5 0 0.8 1 1.0
## 0.01 5 0 0.8 1 1.0
## 0.01 5 0 0.8 3 0.8
## 0.01 5 0 0.8 3 0.8
## 0.01 5 0 0.8 3 1.0
## 0.01 5 0 0.8 3 1.0
## 0.01 5 0 1.0 1 0.8
## 0.01 5 0 1.0 1 0.8
## 0.01 5 0 1.0 1 1.0
## 0.01 5 0 1.0 1 1.0
## 0.01 5 0 1.0 3 0.8
## 0.01 5 0 1.0 3 0.8
## 0.01 5 0 1.0 3 1.0
## 0.01 5 0 1.0 3 1.0
## 0.01 5 1 0.8 1 0.8
## 0.01 5 1 0.8 1 0.8
## 0.01 5 1 0.8 1 1.0
## 0.01 5 1 0.8 1 1.0
## 0.01 5 1 0.8 3 0.8
## 0.01 5 1 0.8 3 0.8
## 0.01 5 1 0.8 3 1.0
## 0.01 5 1 0.8 3 1.0
## 0.01 5 1 1.0 1 0.8
## 0.01 5 1 1.0 1 0.8
## 0.01 5 1 1.0 1 1.0
## 0.01 5 1 1.0 1 1.0
## 0.01 5 1 1.0 3 0.8
## 0.01 5 1 1.0 3 0.8
## 0.01 5 1 1.0 3 1.0
## 0.01 5 1 1.0 3 1.0
## 0.01 5 3 0.8 1 0.8
## 0.01 5 3 0.8 1 0.8
## 0.01 5 3 0.8 1 1.0
## 0.01 5 3 0.8 1 1.0
## 0.01 5 3 0.8 3 0.8
## 0.01 5 3 0.8 3 0.8
## 0.01 5 3 0.8 3 1.0
## 0.01 5 3 0.8 3 1.0
## 0.01 5 3 1.0 1 0.8
## 0.01 5 3 1.0 1 0.8
## 0.01 5 3 1.0 1 1.0
## 0.01 5 3 1.0 1 1.0
## 0.01 5 3 1.0 3 0.8
## 0.01 5 3 1.0 3 0.8
## 0.01 5 3 1.0 3 1.0
## 0.01 5 3 1.0 3 1.0
## 0.01 7 0 0.8 1 0.8
## 0.01 7 0 0.8 1 0.8
## 0.01 7 0 0.8 1 1.0
## 0.01 7 0 0.8 1 1.0
## 0.01 7 0 0.8 3 0.8
## 0.01 7 0 0.8 3 0.8
## 0.01 7 0 0.8 3 1.0
## 0.01 7 0 0.8 3 1.0
## 0.01 7 0 1.0 1 0.8
## 0.01 7 0 1.0 1 0.8
## 0.01 7 0 1.0 1 1.0
## 0.01 7 0 1.0 1 1.0
## 0.01 7 0 1.0 3 0.8
## 0.01 7 0 1.0 3 0.8
## 0.01 7 0 1.0 3 1.0
## 0.01 7 0 1.0 3 1.0
## 0.01 7 1 0.8 1 0.8
## 0.01 7 1 0.8 1 0.8
## 0.01 7 1 0.8 1 1.0
## 0.01 7 1 0.8 1 1.0
## 0.01 7 1 0.8 3 0.8
## 0.01 7 1 0.8 3 0.8
## 0.01 7 1 0.8 3 1.0
## 0.01 7 1 0.8 3 1.0
## 0.01 7 1 1.0 1 0.8
## 0.01 7 1 1.0 1 0.8
## 0.01 7 1 1.0 1 1.0
## 0.01 7 1 1.0 1 1.0
## 0.01 7 1 1.0 3 0.8
## 0.01 7 1 1.0 3 0.8
## 0.01 7 1 1.0 3 1.0
## 0.01 7 1 1.0 3 1.0
## 0.01 7 3 0.8 1 0.8
## 0.01 7 3 0.8 1 0.8
## 0.01 7 3 0.8 1 1.0
## 0.01 7 3 0.8 1 1.0
## 0.01 7 3 0.8 3 0.8
## 0.01 7 3 0.8 3 0.8
## 0.01 7 3 0.8 3 1.0
## 0.01 7 3 0.8 3 1.0
## 0.01 7 3 1.0 1 0.8
## 0.01 7 3 1.0 1 0.8
## 0.01 7 3 1.0 1 1.0
## 0.01 7 3 1.0 1 1.0
## 0.01 7 3 1.0 3 0.8
## 0.01 7 3 1.0 3 0.8
## 0.01 7 3 1.0 3 1.0
## 0.01 7 3 1.0 3 1.0
## 0.10 3 0 0.8 1 0.8
## 0.10 3 0 0.8 1 0.8
## 0.10 3 0 0.8 1 1.0
## 0.10 3 0 0.8 1 1.0
## 0.10 3 0 0.8 3 0.8
## 0.10 3 0 0.8 3 0.8
## 0.10 3 0 0.8 3 1.0
## 0.10 3 0 0.8 3 1.0
## 0.10 3 0 1.0 1 0.8
## 0.10 3 0 1.0 1 0.8
## 0.10 3 0 1.0 1 1.0
## 0.10 3 0 1.0 1 1.0
## 0.10 3 0 1.0 3 0.8
## 0.10 3 0 1.0 3 0.8
## 0.10 3 0 1.0 3 1.0
## 0.10 3 0 1.0 3 1.0
## 0.10 3 1 0.8 1 0.8
## 0.10 3 1 0.8 1 0.8
## 0.10 3 1 0.8 1 1.0
## 0.10 3 1 0.8 1 1.0
## 0.10 3 1 0.8 3 0.8
## 0.10 3 1 0.8 3 0.8
## 0.10 3 1 0.8 3 1.0
## 0.10 3 1 0.8 3 1.0
## 0.10 3 1 1.0 1 0.8
## 0.10 3 1 1.0 1 0.8
## 0.10 3 1 1.0 1 1.0
## 0.10 3 1 1.0 1 1.0
## 0.10 3 1 1.0 3 0.8
## 0.10 3 1 1.0 3 0.8
## 0.10 3 1 1.0 3 1.0
## 0.10 3 1 1.0 3 1.0
## 0.10 3 3 0.8 1 0.8
## 0.10 3 3 0.8 1 0.8
## 0.10 3 3 0.8 1 1.0
## 0.10 3 3 0.8 1 1.0
## 0.10 3 3 0.8 3 0.8
## 0.10 3 3 0.8 3 0.8
## 0.10 3 3 0.8 3 1.0
## 0.10 3 3 0.8 3 1.0
## 0.10 3 3 1.0 1 0.8
## 0.10 3 3 1.0 1 0.8
## 0.10 3 3 1.0 1 1.0
## 0.10 3 3 1.0 1 1.0
## 0.10 3 3 1.0 3 0.8
## 0.10 3 3 1.0 3 0.8
## 0.10 3 3 1.0 3 1.0
## 0.10 3 3 1.0 3 1.0
## 0.10 5 0 0.8 1 0.8
## 0.10 5 0 0.8 1 0.8
## 0.10 5 0 0.8 1 1.0
## 0.10 5 0 0.8 1 1.0
## 0.10 5 0 0.8 3 0.8
## 0.10 5 0 0.8 3 0.8
## 0.10 5 0 0.8 3 1.0
## 0.10 5 0 0.8 3 1.0
## 0.10 5 0 1.0 1 0.8
## 0.10 5 0 1.0 1 0.8
## 0.10 5 0 1.0 1 1.0
## 0.10 5 0 1.0 1 1.0
## 0.10 5 0 1.0 3 0.8
## 0.10 5 0 1.0 3 0.8
## 0.10 5 0 1.0 3 1.0
## 0.10 5 0 1.0 3 1.0
## 0.10 5 1 0.8 1 0.8
## 0.10 5 1 0.8 1 0.8
## 0.10 5 1 0.8 1 1.0
## 0.10 5 1 0.8 1 1.0
## 0.10 5 1 0.8 3 0.8
## 0.10 5 1 0.8 3 0.8
## 0.10 5 1 0.8 3 1.0
## 0.10 5 1 0.8 3 1.0
## 0.10 5 1 1.0 1 0.8
## 0.10 5 1 1.0 1 0.8
## 0.10 5 1 1.0 1 1.0
## 0.10 5 1 1.0 1 1.0
## 0.10 5 1 1.0 3 0.8
## 0.10 5 1 1.0 3 0.8
## 0.10 5 1 1.0 3 1.0
## 0.10 5 1 1.0 3 1.0
## 0.10 5 3 0.8 1 0.8
## 0.10 5 3 0.8 1 0.8
## 0.10 5 3 0.8 1 1.0
## 0.10 5 3 0.8 1 1.0
## 0.10 5 3 0.8 3 0.8
## 0.10 5 3 0.8 3 0.8
## 0.10 5 3 0.8 3 1.0
## 0.10 5 3 0.8 3 1.0
## 0.10 5 3 1.0 1 0.8
## 0.10 5 3 1.0 1 0.8
## 0.10 5 3 1.0 1 1.0
## 0.10 5 3 1.0 1 1.0
## 0.10 5 3 1.0 3 0.8
## 0.10 5 3 1.0 3 0.8
## 0.10 5 3 1.0 3 1.0
## 0.10 5 3 1.0 3 1.0
## 0.10 7 0 0.8 1 0.8
## 0.10 7 0 0.8 1 0.8
## 0.10 7 0 0.8 1 1.0
## 0.10 7 0 0.8 1 1.0
## 0.10 7 0 0.8 3 0.8
## 0.10 7 0 0.8 3 0.8
## 0.10 7 0 0.8 3 1.0
## 0.10 7 0 0.8 3 1.0
## 0.10 7 0 1.0 1 0.8
## 0.10 7 0 1.0 1 0.8
## 0.10 7 0 1.0 1 1.0
## 0.10 7 0 1.0 1 1.0
## 0.10 7 0 1.0 3 0.8
## 0.10 7 0 1.0 3 0.8
## 0.10 7 0 1.0 3 1.0
## 0.10 7 0 1.0 3 1.0
## 0.10 7 1 0.8 1 0.8
## 0.10 7 1 0.8 1 0.8
## 0.10 7 1 0.8 1 1.0
## 0.10 7 1 0.8 1 1.0
## 0.10 7 1 0.8 3 0.8
## 0.10 7 1 0.8 3 0.8
## 0.10 7 1 0.8 3 1.0
## 0.10 7 1 0.8 3 1.0
## 0.10 7 1 1.0 1 0.8
## 0.10 7 1 1.0 1 0.8
## 0.10 7 1 1.0 1 1.0
## 0.10 7 1 1.0 1 1.0
## 0.10 7 1 1.0 3 0.8
## 0.10 7 1 1.0 3 0.8
## 0.10 7 1 1.0 3 1.0
## 0.10 7 1 1.0 3 1.0
## 0.10 7 3 0.8 1 0.8
## 0.10 7 3 0.8 1 0.8
## 0.10 7 3 0.8 1 1.0
## 0.10 7 3 0.8 1 1.0
## 0.10 7 3 0.8 3 0.8
## 0.10 7 3 0.8 3 0.8
## 0.10 7 3 0.8 3 1.0
## 0.10 7 3 0.8 3 1.0
## 0.10 7 3 1.0 1 0.8
## 0.10 7 3 1.0 1 0.8
## 0.10 7 3 1.0 1 1.0
## 0.10 7 3 1.0 1 1.0
## 0.10 7 3 1.0 3 0.8
## 0.10 7 3 1.0 3 0.8
## 0.10 7 3 1.0 3 1.0
## 0.10 7 3 1.0 3 1.0
## 0.30 3 0 0.8 1 0.8
## 0.30 3 0 0.8 1 0.8
## 0.30 3 0 0.8 1 1.0
## 0.30 3 0 0.8 1 1.0
## 0.30 3 0 0.8 3 0.8
## 0.30 3 0 0.8 3 0.8
## 0.30 3 0 0.8 3 1.0
## 0.30 3 0 0.8 3 1.0
## 0.30 3 0 1.0 1 0.8
## 0.30 3 0 1.0 1 0.8
## 0.30 3 0 1.0 1 1.0
## 0.30 3 0 1.0 1 1.0
## 0.30 3 0 1.0 3 0.8
## 0.30 3 0 1.0 3 0.8
## 0.30 3 0 1.0 3 1.0
## 0.30 3 0 1.0 3 1.0
## 0.30 3 1 0.8 1 0.8
## 0.30 3 1 0.8 1 0.8
## 0.30 3 1 0.8 1 1.0
## 0.30 3 1 0.8 1 1.0
## 0.30 3 1 0.8 3 0.8
## 0.30 3 1 0.8 3 0.8
## 0.30 3 1 0.8 3 1.0
## 0.30 3 1 0.8 3 1.0
## 0.30 3 1 1.0 1 0.8
## 0.30 3 1 1.0 1 0.8
## 0.30 3 1 1.0 1 1.0
## 0.30 3 1 1.0 1 1.0
## 0.30 3 1 1.0 3 0.8
## 0.30 3 1 1.0 3 0.8
## 0.30 3 1 1.0 3 1.0
## 0.30 3 1 1.0 3 1.0
## 0.30 3 3 0.8 1 0.8
## 0.30 3 3 0.8 1 0.8
## 0.30 3 3 0.8 1 1.0
## 0.30 3 3 0.8 1 1.0
## 0.30 3 3 0.8 3 0.8
## 0.30 3 3 0.8 3 0.8
## 0.30 3 3 0.8 3 1.0
## 0.30 3 3 0.8 3 1.0
## 0.30 3 3 1.0 1 0.8
## 0.30 3 3 1.0 1 0.8
## 0.30 3 3 1.0 1 1.0
## 0.30 3 3 1.0 1 1.0
## 0.30 3 3 1.0 3 0.8
## 0.30 3 3 1.0 3 0.8
## 0.30 3 3 1.0 3 1.0
## 0.30 3 3 1.0 3 1.0
## 0.30 5 0 0.8 1 0.8
## 0.30 5 0 0.8 1 0.8
## 0.30 5 0 0.8 1 1.0
## 0.30 5 0 0.8 1 1.0
## 0.30 5 0 0.8 3 0.8
## 0.30 5 0 0.8 3 0.8
## 0.30 5 0 0.8 3 1.0
## 0.30 5 0 0.8 3 1.0
## 0.30 5 0 1.0 1 0.8
## 0.30 5 0 1.0 1 0.8
## 0.30 5 0 1.0 1 1.0
## 0.30 5 0 1.0 1 1.0
## 0.30 5 0 1.0 3 0.8
## 0.30 5 0 1.0 3 0.8
## 0.30 5 0 1.0 3 1.0
## 0.30 5 0 1.0 3 1.0
## 0.30 5 1 0.8 1 0.8
## 0.30 5 1 0.8 1 0.8
## 0.30 5 1 0.8 1 1.0
## 0.30 5 1 0.8 1 1.0
## 0.30 5 1 0.8 3 0.8
## 0.30 5 1 0.8 3 0.8
## 0.30 5 1 0.8 3 1.0
## 0.30 5 1 0.8 3 1.0
## 0.30 5 1 1.0 1 0.8
## 0.30 5 1 1.0 1 0.8
## 0.30 5 1 1.0 1 1.0
## 0.30 5 1 1.0 1 1.0
## 0.30 5 1 1.0 3 0.8
## 0.30 5 1 1.0 3 0.8
## 0.30 5 1 1.0 3 1.0
## 0.30 5 1 1.0 3 1.0
## 0.30 5 3 0.8 1 0.8
## 0.30 5 3 0.8 1 0.8
## 0.30 5 3 0.8 1 1.0
## 0.30 5 3 0.8 1 1.0
## 0.30 5 3 0.8 3 0.8
## 0.30 5 3 0.8 3 0.8
## 0.30 5 3 0.8 3 1.0
## 0.30 5 3 0.8 3 1.0
## 0.30 5 3 1.0 1 0.8
## 0.30 5 3 1.0 1 0.8
## 0.30 5 3 1.0 1 1.0
## 0.30 5 3 1.0 1 1.0
## 0.30 5 3 1.0 3 0.8
## 0.30 5 3 1.0 3 0.8
## 0.30 5 3 1.0 3 1.0
## 0.30 5 3 1.0 3 1.0
## 0.30 7 0 0.8 1 0.8
## 0.30 7 0 0.8 1 0.8
## 0.30 7 0 0.8 1 1.0
## 0.30 7 0 0.8 1 1.0
## 0.30 7 0 0.8 3 0.8
## 0.30 7 0 0.8 3 0.8
## 0.30 7 0 0.8 3 1.0
## 0.30 7 0 0.8 3 1.0
## 0.30 7 0 1.0 1 0.8
## 0.30 7 0 1.0 1 0.8
## 0.30 7 0 1.0 1 1.0
## 0.30 7 0 1.0 1 1.0
## 0.30 7 0 1.0 3 0.8
## 0.30 7 0 1.0 3 0.8
## 0.30 7 0 1.0 3 1.0
## 0.30 7 0 1.0 3 1.0
## 0.30 7 1 0.8 1 0.8
## 0.30 7 1 0.8 1 0.8
## 0.30 7 1 0.8 1 1.0
## 0.30 7 1 0.8 1 1.0
## 0.30 7 1 0.8 3 0.8
## 0.30 7 1 0.8 3 0.8
## 0.30 7 1 0.8 3 1.0
## 0.30 7 1 0.8 3 1.0
## 0.30 7 1 1.0 1 0.8
## 0.30 7 1 1.0 1 0.8
## 0.30 7 1 1.0 1 1.0
## 0.30 7 1 1.0 1 1.0
## 0.30 7 1 1.0 3 0.8
## 0.30 7 1 1.0 3 0.8
## 0.30 7 1 1.0 3 1.0
## 0.30 7 1 1.0 3 1.0
## 0.30 7 3 0.8 1 0.8
## 0.30 7 3 0.8 1 0.8
## 0.30 7 3 0.8 1 1.0
## 0.30 7 3 0.8 1 1.0
## 0.30 7 3 0.8 3 0.8
## 0.30 7 3 0.8 3 0.8
## 0.30 7 3 0.8 3 1.0
## 0.30 7 3 0.8 3 1.0
## 0.30 7 3 1.0 1 0.8
## 0.30 7 3 1.0 1 0.8
## 0.30 7 3 1.0 1 1.0
## 0.30 7 3 1.0 1 1.0
## 0.30 7 3 1.0 3 0.8
## 0.30 7 3 1.0 3 0.8
## 0.30 7 3 1.0 3 1.0
## 0.30 7 3 1.0 3 1.0
## nrounds RMSE Rsquared MAE
## 5 11.7494980 0.07740434 11.6897893
## 100 4.6391546 0.27321785 4.5107240
## 5 11.7493499 0.11081278 11.6899118
## 100 4.6368254 0.27234426 4.5101832
## 5 11.7492618 0.13117190 11.6896857
## 100 4.6391525 0.27945962 4.5110111
## 5 11.7494463 0.12685507 11.6900251
## 100 4.6366073 0.27481847 4.5100895
## 5 11.7498127 0.09137458 11.6902094
## 100 4.6390568 0.27381897 4.5118356
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6364097 0.26967898 4.5107000
## 5 11.7498180 0.13544792 11.6902472
## 100 4.6374943 0.27969096 4.5104854
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6357726 0.27261341 4.5101659
## 5 11.7496676 0.10211559 11.6900190
## 100 4.6389340 0.27549264 4.5105117
## 5 11.7495147 0.13060085 11.6900794
## 100 4.6371558 0.26964904 4.5101506
## 5 11.7498325 0.10304278 11.6901709
## 100 4.6396271 0.27599296 4.5109880
## 5 11.7494463 0.13448942 11.6900585
## 100 4.6366694 0.27201715 4.5096876
## 5 11.7494382 0.09953124 11.6898461
## 100 4.6395538 0.27212687 4.5122736
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6364360 0.26909136 4.5105650
## 5 11.7495356 0.09419323 11.6899330
## 100 4.6382914 0.27703210 4.5109506
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6358249 0.27211118 4.5100672
## 5 11.7495982 0.07654492 11.6899085
## 100 4.6386636 0.27198338 4.5095787
## 5 11.7494500 0.10320043 11.6899909
## 100 4.6371165 0.27001446 4.5098623
## 5 11.7498019 0.11405277 11.6901244
## 100 4.6387913 0.27618439 4.5099098
## 5 11.7492666 0.11654898 11.6898160
## 100 4.6362097 0.27386787 4.5091288
## 5 11.7498876 0.07587369 11.6902199
## 100 4.6385693 0.27305280 4.5106146
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6362478 0.26922823 4.5100876
## 5 11.7494203 0.10392826 11.6896953
## 100 4.6377611 0.27277424 4.5098218
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6358995 0.27135231 4.5098144
## 5 11.7499828 0.10637016 11.6903312
## 100 4.6399397 0.27555881 4.5118140
## 5 11.7493184 0.10411394 11.6898623
## 100 4.6368601 0.27799070 4.5109088
## 5 11.7497730 0.09489406 11.6900871
## 100 4.6395841 0.27718576 4.5111724
## 5 11.7493562 0.11274434 11.6899414
## 100 4.6366521 0.27668316 4.5105799
## 5 11.7493139 0.09691938 11.6897174
## 100 4.6383685 0.27514434 4.5113320
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6362089 0.27283317 4.5110799
## 5 11.7499105 0.14510727 11.6903340
## 100 4.6380728 0.27987349 4.5111270
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6357706 0.27637044 4.5108100
## 5 11.7496863 0.10510983 11.6899972
## 100 4.6391088 0.27541604 4.5107071
## 5 11.7494205 0.09385794 11.6899780
## 100 4.6368517 0.27051649 4.5102521
## 5 11.7495528 0.10960676 11.6899065
## 100 4.6390093 0.28356292 4.5110209
## 5 11.7493650 0.12257786 11.6899378
## 100 4.6361225 0.27911151 4.5100819
## 5 11.7493352 0.11496835 11.6897359
## 100 4.6383733 0.27776977 4.5113992
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6361374 0.27328229 4.5108747
## 5 11.7495895 0.15485412 11.6900951
## 100 4.6382242 0.27732494 4.5111866
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6356159 0.27624754 4.5104525
## 5 11.7496487 0.10692271 11.6900170
## 100 4.6406929 0.27415751 4.5119688
## 5 11.7493373 0.11646885 11.6898989
## 100 4.6368828 0.27160861 4.5097915
## 5 11.7495731 0.10105215 11.6899232
## 100 4.6392570 0.27819199 4.5103619
## 5 11.7491502 0.12112250 11.6897334
## 100 4.6362368 0.27547642 4.5095692
## 5 11.7495424 0.13531362 11.6898731
## 100 4.6379391 0.27611479 4.5102997
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6358597 0.27222763 4.5102054
## 5 11.7495966 0.11659344 11.6899678
## 100 4.6383735 0.27339279 4.5106603
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6354985 0.27435554 4.5099298
## 5 11.7496026 0.11704812 11.6899687
## 100 4.6402064 0.27746043 4.5120903
## 5 11.7493946 0.11264406 11.6899457
## 100 4.6368216 0.27698793 4.5108720
## 5 11.7496256 0.15749470 11.6900743
## 100 4.6386279 0.27889155 4.5103544
## 5 11.7492094 0.14706646 11.6897695
## 100 4.6367005 0.27701856 4.5104124
## 5 11.7496864 0.10700205 11.6900642
## 100 4.6387113 0.27472003 4.5116603
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6362061 0.27283431 4.5110772
## 5 11.7498029 0.08587727 11.6902751
## 100 4.6381138 0.27992180 4.5113861
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6357706 0.27637044 4.5108100
## 5 11.7496605 0.11896619 11.6900612
## 100 4.6396521 0.27007720 4.5110915
## 5 11.7494736 0.11602671 11.6900677
## 100 4.6362365 0.27344729 4.5097468
## 5 11.7494910 0.11611858 11.6898296
## 100 4.6388157 0.27901012 4.5103810
## 5 11.7494997 0.11115484 11.6900829
## 100 4.6365128 0.27639391 4.5102030
## 5 11.7495285 0.12251399 11.6898603
## 100 4.6387737 0.27206906 4.5114699
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6361137 0.27332479 4.5108524
## 5 11.7495533 0.10083991 11.6899681
## 100 4.6387154 0.28026005 4.5116132
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6356159 0.27624754 4.5104525
## 5 11.7500008 0.09154627 11.6903256
## 100 4.6393481 0.27190259 4.5105454
## 5 11.7494970 0.12283614 11.6901096
## 100 4.6364893 0.27221005 4.5097306
## 5 11.7495931 0.09667144 11.6898969
## 100 4.6391885 0.27683102 4.5103669
## 5 11.7493037 0.11876822 11.6898698
## 100 4.6360350 0.27519746 4.5092535
## 5 11.7500128 0.09349326 11.6903659
## 100 4.6381398 0.27275593 4.5105728
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6358597 0.27222763 4.5102054
## 5 11.7496328 0.08223125 11.6899643
## 100 4.6377253 0.27604530 4.5101962
## 5 11.7493992 0.08994993 11.6899999
## 100 4.6354985 0.27435554 4.5099298
## 5 7.3528374 0.20482389 7.2641155
## 100 0.9428047 0.36787976 0.7422379
## 5 7.3533656 0.22544101 7.2660520
## 100 0.9486601 0.36084171 0.7457819
## 5 7.3550445 0.18672911 7.2647930
## 100 0.9454164 0.36523323 0.7426580
## 5 7.3518869 0.22072257 7.2647269
## 100 0.9429419 0.36886330 0.7435243
## 5 7.3531573 0.21115735 7.2645612
## 100 0.9434531 0.36868051 0.7416736
## 5 7.3517823 0.23253250 7.2652772
## 100 0.9473240 0.36286695 0.7466159
## 5 7.3544485 0.23060536 7.2664198
## 100 0.9465499 0.36391637 0.7452077
## 5 7.3514828 0.23816839 7.2650398
## 100 0.9443508 0.36671246 0.7452521
## 5 7.3532663 0.19913854 7.2640762
## 100 0.9478911 0.36183902 0.7454192
## 5 7.3507120 0.21859354 7.2634405
## 100 0.9473812 0.36232249 0.7450512
## 5 7.3525154 0.20783616 7.2632700
## 100 0.9455770 0.36594481 0.7444526
## 5 7.3511650 0.23964975 7.2641233
## 100 0.9435887 0.36771850 0.7443631
## 5 7.3548749 0.20846273 7.2666966
## 100 0.9482229 0.36091980 0.7443135
## 5 7.3519577 0.23033416 7.2652852
## 100 0.9455863 0.36457989 0.7453936
## 5 7.3559929 0.23328465 7.2680590
## 100 0.9454358 0.36493203 0.7440074
## 5 7.3516588 0.23647437 7.2650483
## 100 0.9438987 0.36744112 0.7446161
## 5 7.3508741 0.21445377 7.2623883
## 100 0.9485535 0.36077035 0.7474174
## 5 7.3508520 0.23227093 7.2639131
## 100 0.9485001 0.36056428 0.7468176
## 5 7.3510520 0.21062149 7.2618391
## 100 0.9438562 0.36721379 0.7413362
## 5 7.3530315 0.23985710 7.2660301
## 100 0.9403896 0.37278762 0.7405696
## 5 7.3560201 0.22763607 7.2677059
## 100 0.9430299 0.36839250 0.7439065
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9513613 0.35745163 0.7515477
## 5 7.3527223 0.19822739 7.2635920
## 100 0.9466114 0.36342183 0.7458340
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9503111 0.35967077 0.7479946
## 5 7.3554199 0.18396097 7.2658790
## 100 0.9564161 0.35432923 0.7450818
## 5 7.3522255 0.22705676 7.2649324
## 100 0.9479063 0.36359160 0.7410360
## 5 7.3562211 0.19297406 7.2670307
## 100 0.9469368 0.36621658 0.7427649
## 5 7.3529135 0.23503994 7.2658440
## 100 0.9493044 0.36231533 0.7415809
## 5 7.3537160 0.22921512 7.2662192
## 100 0.9511616 0.36049147 0.7397102
## 5 7.3517823 0.23253250 7.2652772
## 100 0.9511574 0.36089084 0.7430282
## 5 7.3575816 0.22029989 7.2688204
## 100 0.9575420 0.35328429 0.7457137
## 5 7.3514828 0.23816839 7.2650398
## 100 0.9499244 0.36209036 0.7439341
## 5 7.3550148 0.19821205 7.2657074
## 100 0.9545613 0.35551047 0.7444723
## 5 7.3528191 0.21814429 7.2652466
## 100 0.9400782 0.37296868 0.7370430
## 5 7.3520337 0.19620364 7.2625246
## 100 0.9498820 0.36189694 0.7449508
## 5 7.3515407 0.23182870 7.2645835
## 100 0.9481968 0.36338380 0.7408602
## 5 7.3553215 0.22880796 7.2672995
## 100 0.9527628 0.35832919 0.7456367
## 5 7.3519577 0.23033416 7.2652852
## 100 0.9424120 0.36912328 0.7384223
## 5 7.3529391 0.21667874 7.2640689
## 100 0.9537793 0.35737365 0.7462401
## 5 7.3516588 0.23647437 7.2650483
## 100 0.9451765 0.36745314 0.7372774
## 5 7.3536180 0.18365210 7.2637284
## 100 0.9411997 0.37100539 0.7399410
## 5 7.3531164 0.23719190 7.2659924
## 100 0.9422393 0.36929825 0.7411609
## 5 7.3531040 0.18489464 7.2631962
## 100 0.9431700 0.36883678 0.7408186
## 5 7.3515902 0.22634257 7.2640504
## 100 0.9441399 0.36633043 0.7408307
## 5 7.3544732 0.23174394 7.2666211
## 100 0.9473425 0.36385882 0.7444427
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9463556 0.36382293 0.7426908
## 5 7.3525752 0.21595258 7.2634092
## 100 0.9441583 0.36792623 0.7385782
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9487993 0.36054672 0.7445112
## 5 7.3538418 0.20391333 7.2648988
## 100 0.9587272 0.35490847 0.7468828
## 5 7.3509630 0.22230397 7.2636934
## 100 0.9635671 0.34624267 0.7492380
## 5 7.3525863 0.21078057 7.2635719
## 100 0.9610144 0.35100169 0.7454040
## 5 7.3512061 0.20672777 7.2637884
## 100 0.9531718 0.35983343 0.7481056
## 5 7.3534034 0.20582523 7.2645721
## 100 0.9678443 0.34334713 0.7519692
## 5 7.3517823 0.23253250 7.2652772
## 100 0.9660695 0.34562129 0.7474841
## 5 7.3554301 0.21297188 7.2669374
## 100 0.9580066 0.35500906 0.7452928
## 5 7.3514828 0.23816839 7.2650398
## 100 0.9597324 0.35223041 0.7507978
## 5 7.3552977 0.20942635 7.2666248
## 100 0.9454523 0.36751049 0.7337913
## 5 7.3512198 0.23814312 7.2642916
## 100 0.9491771 0.36297473 0.7422838
## 5 7.3518888 0.22781524 7.2638228
## 100 0.9551372 0.35709450 0.7425491
## 5 7.3510632 0.22439877 7.2639087
## 100 0.9462421 0.36692946 0.7373856
## 5 7.3554550 0.23079032 7.2678087
## 100 0.9586155 0.35221371 0.7456378
## 5 7.3519577 0.23033416 7.2652852
## 100 0.9485273 0.36224750 0.7398107
## 5 7.3536221 0.21388600 7.2654817
## 100 0.9565683 0.35723918 0.7458403
## 5 7.3516588 0.23647437 7.2650483
## 100 0.9533088 0.35773112 0.7399751
## 5 7.3538492 0.19423176 7.2642930
## 100 0.9481140 0.36287010 0.7409248
## 5 7.3520559 0.23282997 7.2647923
## 100 0.9432489 0.36856371 0.7380205
## 5 7.3553160 0.22437083 7.2669433
## 100 0.9435104 0.36892142 0.7389938
## 5 7.3507045 0.22876496 7.2636031
## 100 0.9505445 0.35878001 0.7440750
## 5 7.3545356 0.20916759 7.2658461
## 100 0.9412705 0.37232629 0.7385716
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9428100 0.36886997 0.7397426
## 5 7.3551725 0.21898143 7.2668561
## 100 0.9386839 0.37501786 0.7327559
## 5 7.3514000 0.23667186 7.2646346
## 100 0.9415948 0.36993006 0.7379886
## 5 2.3290085 0.27058085 2.0871099
## 100 0.9807494 0.33571363 0.7547568
## 5 2.3211838 0.26975722 2.0818811
## 100 0.9772689 0.33675423 0.7612742
## 5 2.3186803 0.27542903 2.0775348
## 100 0.9811673 0.33530793 0.7656917
## 5 2.3183613 0.28535168 2.0809648
## 100 0.9615118 0.35212032 0.7530860
## 5 2.3269634 0.25962717 2.0851538
## 100 0.9743232 0.34060678 0.7606105
## 5 2.3214108 0.29251519 2.0870296
## 100 0.9678641 0.34621723 0.7569181
## 5 2.3301098 0.26018883 2.0887404
## 100 0.9867931 0.32703655 0.7643883
## 5 2.3210805 0.29362546 2.0868893
## 100 0.9689197 0.34370394 0.7551272
## 5 2.3277664 0.26902294 2.0867419
## 100 0.9693145 0.34667154 0.7562780
## 5 2.3239112 0.28695147 2.0872726
## 100 0.9641565 0.34821817 0.7553384
## 5 2.3334239 0.26568285 2.0886109
## 100 0.9636921 0.35070142 0.7468970
## 5 2.3260304 0.27321746 2.0872491
## 100 0.9651664 0.34702462 0.7542961
## 5 2.3250797 0.26207959 2.0830861
## 100 0.9714091 0.34241211 0.7519374
## 5 2.3215987 0.29320142 2.0874349
## 100 0.9597409 0.35006910 0.7527654
## 5 2.3290939 0.27316662 2.0884865
## 100 0.9661536 0.35080762 0.7600647
## 5 2.3212054 0.29435333 2.0872257
## 100 0.9586428 0.35292399 0.7527586
## 5 2.3231028 0.27439007 2.0821750
## 100 0.9679045 0.34388579 0.7567852
## 5 2.3289913 0.26951646 2.0890706
## 100 0.9529743 0.35575091 0.7504652
## 5 2.3255064 0.25297130 2.0790709
## 100 0.9601993 0.35112645 0.7506044
## 5 2.3196500 0.28663493 2.0810304
## 100 0.9560167 0.35243057 0.7487321
## 5 2.3255362 0.26890447 2.0850048
## 100 0.9694091 0.34199280 0.7604642
## 5 2.3207596 0.28663923 2.0854221
## 100 0.9527549 0.35407891 0.7515966
## 5 2.3280319 0.27817684 2.0886915
## 100 0.9730782 0.33628719 0.7580379
## 5 2.3203663 0.28779114 2.0852129
## 100 0.9620411 0.34331457 0.7533385
## 5 2.3266259 0.27778523 2.0878182
## 100 1.0213717 0.30196758 0.7858523
## 5 2.3176590 0.28581152 2.0824013
## 100 0.9983731 0.31967018 0.7777987
## 5 2.3256793 0.29301417 2.0919115
## 100 1.0157683 0.30932004 0.7902023
## 5 2.3237905 0.29906639 2.0910332
## 100 0.9981650 0.31899296 0.7729769
## 5 2.3319631 0.28118826 2.0980084
## 100 1.0118510 0.31483512 0.7825935
## 5 2.3240279 0.28826447 2.0937522
## 100 1.0140551 0.30461821 0.7887163
## 5 2.3271020 0.29183368 2.0955712
## 100 1.0211173 0.30331210 0.7862692
## 5 2.3187088 0.30106624 2.0908690
## 100 0.9983463 0.32145055 0.7731137
## 5 2.3292686 0.29289859 2.0957779
## 100 1.0005144 0.31560752 0.7807425
## 5 2.3177345 0.29370523 2.0822386
## 100 0.9800780 0.33327141 0.7620533
## 5 2.3334875 0.27918430 2.0952135
## 100 1.0097131 0.31067332 0.7842428
## 5 2.3162577 0.29518121 2.0847007
## 100 0.9765861 0.33617609 0.7599016
## 5 2.3228781 0.27737750 2.0876965
## 100 0.9980598 0.32094518 0.7768272
## 5 2.3199395 0.29336988 2.0903969
## 100 0.9673477 0.34626773 0.7536354
## 5 2.3318505 0.28558239 2.0992370
## 100 0.9947164 0.32612115 0.7694702
## 5 2.3183597 0.30205656 2.0902894
## 100 0.9689379 0.34174255 0.7529529
## 5 2.3308099 0.28537186 2.0954507
## 100 0.9659155 0.34780366 0.7537462
## 5 2.3281885 0.28582639 2.0912271
## 100 0.9617952 0.34701695 0.7512393
## 5 2.3313626 0.27817107 2.0939915
## 100 0.9658659 0.35022457 0.7552286
## 5 2.3197297 0.29420726 2.0869300
## 100 0.9576736 0.35106617 0.7468941
## 5 2.3278045 0.27517500 2.0918246
## 100 0.9654673 0.35005150 0.7528583
## 5 2.3148446 0.29347104 2.0834364
## 100 0.9627346 0.34361461 0.7509519
## 5 2.3280816 0.28153633 2.0932707
## 100 0.9735622 0.34082259 0.7615553
## 5 2.3184829 0.29309473 2.0875213
## 100 0.9583072 0.34927710 0.7494729
## 5 2.3350786 0.27257270 2.0998032
## 100 1.0531830 0.27294183 0.8167890
## 5 2.3269757 0.30399474 2.1008050
## 100 1.0174545 0.30415257 0.7883645
## 5 2.3294180 0.27704682 2.0921520
## 100 1.0339639 0.29357216 0.7981258
## 5 2.3270274 0.28676148 2.0943206
## 100 1.0020465 0.32187734 0.7772423
## 5 2.3366809 0.27912342 2.1057075
## 100 1.0380983 0.29203073 0.8063551
## 5 2.3263340 0.28887536 2.0974577
## 100 1.0302093 0.29021769 0.8021509
## 5 2.3322897 0.30723572 2.1048086
## 100 1.0529501 0.27901996 0.8173337
## 5 2.3225308 0.29994883 2.0959503
## 100 1.0157979 0.30270643 0.7960660
## 5 2.3372716 0.26293625 2.0987800
## 100 1.0143647 0.30023348 0.7890216
## 5 2.3247363 0.28560350 2.0917763
## 100 0.9790514 0.33391773 0.7614866
## 5 2.3334361 0.27034903 2.0952088
## 100 1.0156584 0.30068983 0.7984868
## 5 2.3197936 0.29550787 2.0878880
## 100 0.9858484 0.32818729 0.7712411
## 5 2.3291507 0.28676470 2.0977460
## 100 1.0271217 0.29481285 0.7935624
## 5 2.3199981 0.29575529 2.0913360
## 100 0.9786451 0.33242072 0.7611091
## 5 2.3260863 0.29177178 2.0965048
## 100 1.0176273 0.30655444 0.7915314
## 5 2.3195073 0.30693754 2.0933785
## 100 0.9817435 0.32860308 0.7622255
## 5 2.3289346 0.26848692 2.0909054
## 100 0.9676350 0.34769607 0.7542693
## 5 2.3256535 0.26810503 2.0876490
## 100 0.9552859 0.35675179 0.7501941
## 5 2.3260407 0.27099568 2.0858368
## 100 0.9685659 0.34612766 0.7560081
## 5 2.3170527 0.29360498 2.0834059
## 100 0.9669772 0.34190677 0.7530561
## 5 2.3159564 0.29056738 2.0825692
## 100 0.9726918 0.34350952 0.7569778
## 5 2.3152445 0.29509314 2.0849202
## 100 0.9551702 0.35503605 0.7450100
## 5 2.3259640 0.29686713 2.0939425
## 100 0.9808185 0.33245953 0.7644920
## 5 2.3198609 0.29486809 2.0897090
## 100 0.9559349 0.35400751 0.7488271
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were nrounds = 100, max_depth = 7, eta
## = 0.1, gamma = 3, colsample_bytree = 1, min_child_weight = 3 and subsample
## = 0.8.
# Hacer predicciones en el conjunto de prueba
test_results$xgb <- predict(xgb_tune, testing)
# Evaluar el modelo Gradient Boosting
xgb_metrics <- postResample(pred = test_results$xgb, obs = testing$average_grade)
print(xgb_metrics)## RMSE Rsquared MAE
## 0.9851380 0.3585118 0.7387951
The Gradient Boosting model outperforms others in terms of predictive accuracy and will be used as the final model for our prediction task. This aligns with the power of machine learning to capture complex relationships in data. Its RMSE (0.984) is relatively lower compared to other models, indicating that it makes predictions with smaller errors. Its R-squared (0.359) means that the model explains around 35.9% of the variance in average_grade, which is better than many of the other models we tested. On its part, the MAE (0.735) is also lower.
The model identifies academic performance metrics (average evaluations, admission grades, and prior grades) as the strongest predictors of outcomes, aligning with expectations in educational models. Socioeconomic factors like GDP and unemployment rates show moderate influence, highlighting the role of broader economic conditions. Demographic variables such as age at enrollment and parental education (e.g., mother’s higher education) also contribute meaningfully. In contrast, variables related to some maternal occupations or special needs show negligible impact.
Neural Network
set.seed(123)
# Definir la grilla de hiperparámetros (ajustar número de neuronas)
nn_grid <- expand.grid(size = c(2, 4, 6), # Número de neuronas en la capa oculta
decay = c(0.1, 0.01)) # Regularización para evitar overfitting
# Entrenar la red neuronal
set.seed(123)
nn_tune <- train(average_grade ~ .,
data = training,
method = "nnet",
preProc = c('scale', 'center'),
tuneGrid = nn_grid,
maxit = 200, # Iteraciones limitadas para evitar largos tiempos
trControl = ctrl,
linout = TRUE) # Salida lineal porque estamos haciendo regresión## # weights: 87
## initial value 237820.476727
## iter 10 value 2350.776089
## iter 20 value 2307.400647
## iter 30 value 2222.196323
## iter 40 value 2114.013174
## iter 50 value 2014.907656
## iter 60 value 1913.347050
## iter 70 value 1844.628785
## iter 80 value 1816.883577
## iter 90 value 1809.232026
## iter 100 value 1785.296106
## iter 110 value 1717.212111
## iter 120 value 1635.527669
## iter 130 value 1579.270292
## iter 140 value 1496.576090
## iter 150 value 1443.228012
## iter 160 value 1385.396615
## iter 170 value 1358.334664
## iter 180 value 1342.230280
## iter 190 value 1317.233333
## iter 200 value 1304.783574
## final value 1304.783574
## stopped after 200 iterations
## # weights: 173
## initial value 272020.366490
## iter 10 value 14140.578871
## iter 20 value 2586.976644
## iter 30 value 2447.584669
## iter 40 value 2340.722382
## iter 50 value 2294.850819
## iter 60 value 2229.936361
## iter 70 value 2139.181571
## iter 80 value 2058.999793
## iter 90 value 2012.469875
## iter 100 value 1985.341327
## iter 110 value 1969.025601
## iter 120 value 1947.807048
## iter 130 value 1917.750641
## iter 140 value 1897.801901
## iter 150 value 1859.339278
## iter 160 value 1726.388297
## iter 170 value 1620.394228
## iter 180 value 1551.739888
## iter 190 value 1481.964652
## iter 200 value 1414.553048
## final value 1414.553048
## stopped after 200 iterations
## # weights: 259
## initial value 249723.826980
## iter 10 value 1645.230129
## iter 20 value 1357.904984
## iter 30 value 1265.581212
## iter 40 value 1194.048116
## iter 50 value 1159.281293
## iter 60 value 1140.860770
## iter 70 value 1116.086248
## iter 80 value 1098.646222
## iter 90 value 1086.190795
## iter 100 value 1080.162657
## iter 110 value 1077.269401
## iter 120 value 1074.127665
## iter 130 value 1070.875735
## iter 140 value 1069.584315
## iter 150 value 1067.510445
## iter 160 value 1062.956209
## iter 170 value 1057.827144
## iter 180 value 1050.853020
## iter 190 value 1047.726246
## iter 200 value 1045.553845
## final value 1045.553845
## stopped after 200 iterations
## # weights: 87
## initial value 247351.910950
## iter 10 value 2188.216653
## iter 20 value 2104.266129
## iter 30 value 2051.225336
## iter 40 value 1963.559750
## iter 50 value 1878.905130
## iter 60 value 1863.703167
## iter 70 value 1841.497989
## iter 80 value 1818.649999
## iter 90 value 1789.338489
## iter 100 value 1772.260091
## iter 110 value 1764.923482
## iter 120 value 1757.185815
## iter 130 value 1751.193703
## iter 140 value 1743.099645
## iter 150 value 1741.368658
## iter 160 value 1740.097575
## iter 170 value 1684.024465
## iter 180 value 1656.054353
## iter 190 value 1645.448354
## iter 200 value 1625.682017
## final value 1625.682017
## stopped after 200 iterations
## # weights: 173
## initial value 245397.739162
## iter 10 value 2124.699309
## iter 20 value 1583.050833
## iter 30 value 1443.665599
## iter 40 value 1396.240539
## iter 50 value 1337.975729
## iter 60 value 1300.679313
## iter 70 value 1281.014032
## iter 80 value 1270.360210
## iter 90 value 1264.105859
## iter 100 value 1260.264130
## iter 110 value 1256.396599
## iter 120 value 1251.006388
## iter 130 value 1245.000352
## iter 140 value 1239.865844
## iter 150 value 1234.558365
## iter 160 value 1230.778533
## iter 170 value 1225.556187
## iter 180 value 1220.505837
## iter 190 value 1216.879496
## iter 200 value 1214.299603
## final value 1214.299603
## stopped after 200 iterations
## # weights: 259
## initial value 259297.361350
## iter 10 value 1441.176752
## iter 20 value 1268.686374
## iter 30 value 1221.814236
## iter 40 value 1175.745084
## iter 50 value 1135.875088
## iter 60 value 1101.100467
## iter 70 value 1062.375339
## iter 80 value 1041.393113
## iter 90 value 1025.091171
## iter 100 value 1009.718958
## iter 110 value 1000.334526
## iter 120 value 992.393380
## iter 130 value 983.359363
## iter 140 value 976.988724
## iter 150 value 971.512187
## iter 160 value 966.540041
## iter 170 value 962.286217
## iter 180 value 959.860263
## iter 190 value 957.786671
## iter 200 value 955.250563
## final value 955.250563
## stopped after 200 iterations
## # weights: 87
## initial value 252089.389752
## iter 10 value 2220.349465
## iter 20 value 2178.479354
## iter 30 value 2141.802450
## iter 40 value 2096.421084
## iter 50 value 1922.436634
## iter 60 value 1858.598184
## iter 70 value 1814.627925
## iter 80 value 1740.489927
## iter 90 value 1667.817547
## iter 100 value 1630.577802
## iter 110 value 1612.953772
## iter 120 value 1591.423945
## iter 130 value 1571.396719
## iter 140 value 1560.629937
## iter 150 value 1554.959080
## iter 160 value 1536.871902
## iter 170 value 1512.852581
## iter 180 value 1429.985235
## iter 190 value 1388.007037
## iter 200 value 1358.155131
## final value 1358.155131
## stopped after 200 iterations
## # weights: 173
## initial value 268222.067869
## iter 10 value 2017.809868
## iter 20 value 1496.272958
## iter 30 value 1394.838091
## iter 40 value 1334.802744
## iter 50 value 1290.859880
## iter 60 value 1258.046094
## iter 70 value 1238.153369
## iter 80 value 1230.498686
## iter 90 value 1224.655796
## iter 100 value 1217.917434
## iter 110 value 1210.842548
## iter 120 value 1203.964221
## iter 130 value 1196.056167
## iter 140 value 1193.399792
## iter 150 value 1191.975481
## iter 160 value 1191.617604
## iter 170 value 1190.872172
## iter 180 value 1187.178740
## iter 190 value 1182.310925
## iter 200 value 1179.891672
## final value 1179.891672
## stopped after 200 iterations
## # weights: 259
## initial value 260198.182216
## iter 10 value 1600.029487
## iter 20 value 1315.603332
## iter 30 value 1246.203859
## iter 40 value 1190.232102
## iter 50 value 1163.203807
## iter 60 value 1116.262583
## iter 70 value 1082.902790
## iter 80 value 1067.313988
## iter 90 value 1059.822585
## iter 100 value 1053.378526
## iter 110 value 1046.612232
## iter 120 value 1039.780076
## iter 130 value 1034.886333
## iter 140 value 1031.577033
## iter 150 value 1029.244952
## iter 160 value 1027.368257
## iter 170 value 1025.122347
## iter 180 value 1022.424781
## iter 190 value 1018.604212
## iter 200 value 1015.540256
## final value 1015.540256
## stopped after 200 iterations
## # weights: 87
## initial value 245500.807629
## iter 10 value 2441.121626
## iter 20 value 2337.212077
## iter 30 value 2159.243331
## iter 40 value 1952.834989
## iter 50 value 1841.530432
## iter 60 value 1710.189187
## iter 70 value 1633.370643
## iter 80 value 1591.236324
## iter 90 value 1571.402909
## iter 100 value 1554.634268
## iter 110 value 1546.349334
## iter 120 value 1542.797765
## iter 130 value 1541.504785
## iter 140 value 1539.625949
## iter 150 value 1533.089646
## iter 160 value 1523.142903
## iter 170 value 1518.090889
## iter 180 value 1515.509707
## iter 190 value 1512.132403
## iter 200 value 1507.775706
## final value 1507.775706
## stopped after 200 iterations
## # weights: 173
## initial value 276192.273080
## iter 10 value 1884.662108
## iter 20 value 1438.575377
## iter 30 value 1322.376980
## iter 40 value 1269.389723
## iter 50 value 1239.457192
## iter 60 value 1231.151011
## iter 70 value 1215.776984
## iter 80 value 1207.624675
## iter 90 value 1202.614993
## iter 100 value 1196.010951
## iter 110 value 1191.964983
## iter 120 value 1187.031442
## iter 130 value 1183.120260
## iter 140 value 1181.088167
## iter 150 value 1179.653466
## iter 160 value 1177.342967
## iter 170 value 1174.835879
## iter 180 value 1170.189252
## iter 190 value 1165.503831
## iter 200 value 1159.603489
## final value 1159.603489
## stopped after 200 iterations
## # weights: 259
## initial value 258118.429694
## iter 10 value 1446.279559
## iter 20 value 1243.998367
## iter 30 value 1168.938693
## iter 40 value 1127.331725
## iter 50 value 1104.300353
## iter 60 value 1071.178682
## iter 70 value 1037.550913
## iter 80 value 1011.041939
## iter 90 value 994.592888
## iter 100 value 986.779261
## iter 110 value 980.878100
## iter 120 value 977.174959
## iter 130 value 974.817231
## iter 140 value 972.472102
## iter 150 value 969.654841
## iter 160 value 966.953613
## iter 170 value 963.711371
## iter 180 value 960.896485
## iter 190 value 957.827469
## iter 200 value 954.957488
## final value 954.957488
## stopped after 200 iterations
## # weights: 87
## initial value 239449.230768
## iter 10 value 2015.203617
## iter 20 value 1963.922072
## iter 30 value 1914.622217
## iter 40 value 1759.141871
## iter 50 value 1713.601194
## iter 60 value 1652.785027
## iter 70 value 1601.272229
## iter 80 value 1575.862981
## iter 90 value 1561.270704
## iter 100 value 1555.267244
## iter 110 value 1541.851470
## iter 120 value 1498.533110
## iter 130 value 1458.864817
## iter 140 value 1436.298359
## iter 150 value 1417.064501
## iter 160 value 1364.512321
## iter 170 value 1346.758687
## iter 180 value 1331.674699
## iter 190 value 1324.120797
## iter 200 value 1317.982970
## final value 1317.982970
## stopped after 200 iterations
## # weights: 173
## initial value 245460.173350
## iter 10 value 1788.938170
## iter 20 value 1473.882105
## iter 30 value 1426.309834
## iter 40 value 1386.116250
## iter 50 value 1362.345768
## iter 60 value 1342.886382
## iter 70 value 1333.825071
## iter 80 value 1320.850176
## iter 90 value 1312.695213
## iter 100 value 1305.092014
## iter 110 value 1301.534319
## iter 120 value 1299.006480
## iter 130 value 1294.031689
## iter 140 value 1284.020808
## iter 150 value 1280.351500
## iter 160 value 1278.046653
## iter 170 value 1275.093658
## iter 180 value 1268.998793
## iter 190 value 1261.136461
## iter 200 value 1253.504330
## final value 1253.504330
## stopped after 200 iterations
## # weights: 259
## initial value 226054.802229
## iter 10 value 1570.498344
## iter 20 value 1268.566659
## iter 30 value 1187.861584
## iter 40 value 1153.734069
## iter 50 value 1132.926187
## iter 60 value 1105.725927
## iter 70 value 1071.772476
## iter 80 value 1048.027395
## iter 90 value 1032.943423
## iter 100 value 1023.500592
## iter 110 value 1018.258695
## iter 120 value 1014.735057
## iter 130 value 1010.016251
## iter 140 value 1006.520919
## iter 150 value 1004.453549
## iter 160 value 1002.881994
## iter 170 value 1001.134920
## iter 180 value 998.916604
## iter 190 value 995.700146
## iter 200 value 993.178113
## final value 993.178113
## stopped after 200 iterations
## # weights: 87
## initial value 269674.949016
## iter 10 value 2145.121641
## iter 20 value 2101.036135
## iter 30 value 2085.533612
## iter 40 value 2013.080709
## iter 50 value 1919.022266
## iter 60 value 1854.370960
## iter 70 value 1773.684543
## iter 80 value 1668.523872
## iter 90 value 1608.029034
## iter 100 value 1548.082536
## iter 110 value 1499.356499
## iter 120 value 1480.985152
## iter 130 value 1457.078124
## iter 140 value 1440.792365
## iter 150 value 1430.319135
## iter 160 value 1419.636178
## iter 170 value 1407.691265
## iter 180 value 1395.316192
## iter 190 value 1372.953193
## iter 200 value 1344.889080
## final value 1344.889080
## stopped after 200 iterations
## # weights: 173
## initial value 232511.534163
## iter 10 value 3262.145541
## iter 20 value 1737.067937
## iter 30 value 1493.282543
## iter 40 value 1406.532064
## iter 50 value 1371.547193
## iter 60 value 1340.250028
## iter 70 value 1320.942184
## iter 80 value 1316.565206
## iter 90 value 1315.314640
## iter 100 value 1314.133404
## iter 110 value 1308.155464
## iter 120 value 1303.804910
## iter 130 value 1302.074494
## iter 140 value 1300.084899
## iter 150 value 1299.296112
## iter 160 value 1298.054783
## iter 170 value 1295.758155
## iter 180 value 1290.193409
## iter 190 value 1284.118271
## iter 200 value 1282.676418
## final value 1282.676418
## stopped after 200 iterations
## # weights: 259
## initial value 273981.334440
## iter 10 value 1466.560275
## iter 20 value 1275.372495
## iter 30 value 1196.340932
## iter 40 value 1147.463694
## iter 50 value 1112.559473
## iter 60 value 1073.232401
## iter 70 value 1034.663065
## iter 80 value 1013.462550
## iter 90 value 1004.743042
## iter 100 value 995.531306
## iter 110 value 990.078363
## iter 120 value 986.947244
## iter 130 value 984.262752
## iter 140 value 982.009694
## iter 150 value 980.189060
## iter 160 value 977.862201
## iter 170 value 974.492496
## iter 180 value 971.301404
## iter 190 value 969.159206
## iter 200 value 966.930527
## final value 966.930527
## stopped after 200 iterations
## # weights: 87
## initial value 251756.678049
## iter 10 value 2828.919503
## iter 20 value 2455.233281
## iter 30 value 2073.352030
## iter 40 value 1996.294507
## iter 50 value 1861.676026
## iter 60 value 1718.189396
## iter 70 value 1648.411727
## iter 80 value 1614.921901
## iter 90 value 1588.322994
## iter 100 value 1566.930732
## iter 110 value 1547.116455
## iter 120 value 1524.399694
## iter 130 value 1512.264983
## iter 140 value 1466.479393
## iter 150 value 1458.393637
## iter 160 value 1438.882618
## iter 170 value 1422.177239
## iter 180 value 1405.042327
## iter 190 value 1385.798708
## iter 200 value 1381.063597
## final value 1381.063597
## stopped after 200 iterations
## # weights: 173
## initial value 267946.605236
## iter 10 value 1794.443069
## iter 20 value 1513.860004
## iter 30 value 1432.393961
## iter 40 value 1349.232332
## iter 50 value 1307.764978
## iter 60 value 1295.675083
## iter 70 value 1279.105575
## iter 80 value 1263.461643
## iter 90 value 1253.327400
## iter 100 value 1242.446914
## iter 110 value 1229.985926
## iter 120 value 1221.029249
## iter 130 value 1210.435666
## iter 140 value 1190.701795
## iter 150 value 1181.065343
## iter 160 value 1175.671164
## iter 170 value 1171.748686
## iter 180 value 1170.415472
## iter 190 value 1169.878437
## iter 200 value 1168.940392
## final value 1168.940392
## stopped after 200 iterations
## # weights: 259
## initial value 303576.752333
## iter 10 value 1635.313347
## iter 20 value 1297.547085
## iter 30 value 1219.422293
## iter 40 value 1185.093431
## iter 50 value 1139.887136
## iter 60 value 1110.168042
## iter 70 value 1092.016044
## iter 80 value 1077.889590
## iter 90 value 1068.895721
## iter 100 value 1061.059341
## iter 110 value 1053.871925
## iter 120 value 1050.683613
## iter 130 value 1048.428781
## iter 140 value 1046.294318
## iter 150 value 1044.559974
## iter 160 value 1043.220123
## iter 170 value 1042.164915
## iter 180 value 1041.488410
## iter 190 value 1039.899955
## iter 200 value 1038.568110
## final value 1038.568110
## stopped after 200 iterations
## # weights: 87
## initial value 241503.896535
## iter 10 value 2173.702744
## iter 20 value 2142.718542
## iter 30 value 2130.913977
## iter 40 value 2094.206441
## iter 50 value 1922.691106
## iter 60 value 1783.475278
## iter 70 value 1682.391303
## iter 80 value 1512.014307
## iter 90 value 1438.447168
## iter 100 value 1369.548192
## iter 110 value 1347.116993
## iter 120 value 1332.706132
## iter 130 value 1320.424805
## iter 140 value 1311.334061
## iter 150 value 1302.578738
## iter 160 value 1298.273975
## iter 170 value 1295.097276
## iter 180 value 1293.823865
## iter 190 value 1293.072467
## iter 200 value 1292.477825
## final value 1292.477825
## stopped after 200 iterations
## # weights: 173
## initial value 250355.694192
## iter 10 value 2471.166292
## iter 20 value 2097.779226
## iter 30 value 1990.682592
## iter 40 value 1814.986403
## iter 50 value 1650.097076
## iter 60 value 1555.623909
## iter 70 value 1502.498734
## iter 80 value 1448.374533
## iter 90 value 1399.953627
## iter 100 value 1372.904718
## iter 110 value 1348.023042
## iter 120 value 1336.047892
## iter 130 value 1326.113045
## iter 140 value 1319.352085
## iter 150 value 1312.207475
## iter 160 value 1306.926650
## iter 170 value 1299.882361
## iter 180 value 1297.862747
## iter 190 value 1293.468985
## iter 200 value 1288.611913
## final value 1288.611913
## stopped after 200 iterations
## # weights: 259
## initial value 220631.032001
## iter 10 value 1600.988431
## iter 20 value 1328.260261
## iter 30 value 1245.623342
## iter 40 value 1199.725354
## iter 50 value 1165.181119
## iter 60 value 1135.322706
## iter 70 value 1110.401778
## iter 80 value 1074.953849
## iter 90 value 1048.398917
## iter 100 value 1033.603186
## iter 110 value 1012.160702
## iter 120 value 997.659510
## iter 130 value 991.228386
## iter 140 value 985.438841
## iter 150 value 979.213425
## iter 160 value 976.362076
## iter 170 value 974.225868
## iter 180 value 970.479541
## iter 190 value 967.588085
## iter 200 value 965.855965
## final value 965.855965
## stopped after 200 iterations
## # weights: 87
## initial value 255107.659860
## iter 10 value 2071.264018
## iter 20 value 1990.028777
## iter 30 value 1876.394882
## iter 40 value 1790.315420
## iter 50 value 1700.008958
## iter 60 value 1642.295744
## iter 70 value 1562.503753
## iter 80 value 1520.163748
## iter 90 value 1485.304011
## iter 100 value 1476.298663
## iter 110 value 1470.438779
## iter 120 value 1456.756679
## iter 130 value 1389.234649
## iter 140 value 1344.382082
## iter 150 value 1328.610911
## iter 160 value 1303.124079
## iter 170 value 1288.736257
## iter 180 value 1276.888112
## iter 190 value 1272.355517
## iter 200 value 1268.834302
## final value 1268.834302
## stopped after 200 iterations
## # weights: 173
## initial value 267432.727539
## iter 10 value 1630.861880
## iter 20 value 1459.563052
## iter 30 value 1398.331807
## iter 40 value 1339.890858
## iter 50 value 1308.854068
## iter 60 value 1283.337485
## iter 70 value 1266.050060
## iter 80 value 1253.116168
## iter 90 value 1238.457179
## iter 100 value 1231.093568
## iter 110 value 1224.801055
## iter 120 value 1217.470703
## iter 130 value 1210.591347
## iter 140 value 1203.767299
## iter 150 value 1198.191560
## iter 160 value 1192.279000
## iter 170 value 1187.878779
## iter 180 value 1182.790572
## iter 190 value 1177.564684
## iter 200 value 1174.429874
## final value 1174.429874
## stopped after 200 iterations
## # weights: 259
## initial value 304652.783737
## iter 10 value 1437.478914
## iter 20 value 1261.071802
## iter 30 value 1212.419023
## iter 40 value 1162.944852
## iter 50 value 1126.551319
## iter 60 value 1101.771491
## iter 70 value 1083.711558
## iter 80 value 1068.962376
## iter 90 value 1053.304585
## iter 100 value 1040.833914
## iter 110 value 1029.941422
## iter 120 value 1019.109574
## iter 130 value 1012.370254
## iter 140 value 1008.733116
## iter 150 value 1004.662377
## iter 160 value 1001.277303
## iter 170 value 999.366120
## iter 180 value 997.982917
## iter 190 value 996.568826
## iter 200 value 995.596940
## final value 995.596940
## stopped after 200 iterations
## # weights: 87
## initial value 254155.103841
## iter 10 value 2177.360706
## iter 20 value 2083.383978
## iter 30 value 2042.493458
## iter 40 value 2015.911443
## iter 50 value 1854.797340
## iter 60 value 1663.456041
## iter 70 value 1616.411290
## iter 80 value 1587.460817
## iter 90 value 1571.387368
## iter 100 value 1545.130348
## iter 110 value 1498.208419
## iter 120 value 1453.184569
## iter 130 value 1427.596342
## iter 140 value 1360.623977
## iter 150 value 1319.305591
## iter 160 value 1313.183928
## iter 170 value 1309.018775
## iter 180 value 1308.730494
## iter 190 value 1305.344569
## iter 200 value 1288.631874
## final value 1288.631874
## stopped after 200 iterations
## # weights: 173
## initial value 267362.094474
## iter 10 value 5934.367918
## iter 20 value 3072.810546
## iter 30 value 2611.974463
## iter 40 value 2343.330082
## iter 50 value 2280.269856
## iter 60 value 2223.018469
## iter 70 value 2184.823580
## iter 80 value 2163.351487
## iter 90 value 2151.128334
## iter 100 value 2128.312826
## iter 110 value 2089.519203
## iter 120 value 2022.758222
## iter 130 value 1984.908964
## iter 140 value 1945.077959
## iter 150 value 1905.632525
## iter 160 value 1859.549025
## iter 170 value 1742.374718
## iter 180 value 1643.362453
## iter 190 value 1553.737204
## iter 200 value 1466.304537
## final value 1466.304537
## stopped after 200 iterations
## # weights: 259
## initial value 231741.865050
## iter 10 value 1484.027792
## iter 20 value 1232.314903
## iter 30 value 1171.634569
## iter 40 value 1136.654218
## iter 50 value 1115.013913
## iter 60 value 1098.825998
## iter 70 value 1085.033384
## iter 80 value 1056.603050
## iter 90 value 1036.976878
## iter 100 value 1023.432747
## iter 110 value 1014.892193
## iter 120 value 1008.198477
## iter 130 value 1002.499916
## iter 140 value 998.467955
## iter 150 value 993.511577
## iter 160 value 990.175011
## iter 170 value 987.430165
## iter 180 value 984.292670
## iter 190 value 981.713162
## iter 200 value 979.893841
## final value 979.893841
## stopped after 200 iterations
## # weights: 87
## initial value 312724.618334
## iter 10 value 3069.062659
## iter 20 value 2903.729961
## iter 30 value 2720.469893
## iter 40 value 2513.434131
## iter 50 value 2348.812804
## iter 60 value 2230.006055
## iter 70 value 2127.214669
## iter 80 value 2034.021144
## iter 90 value 1960.283168
## iter 100 value 1871.046436
## iter 110 value 1832.036363
## iter 120 value 1811.569464
## iter 130 value 1792.748090
## iter 140 value 1749.870891
## iter 150 value 1711.509518
## iter 160 value 1703.073659
## iter 170 value 1691.191650
## iter 180 value 1686.121948
## iter 190 value 1685.393017
## iter 200 value 1684.541189
## final value 1684.541189
## stopped after 200 iterations
## Neural Network
##
## 1938 samples
## 41 predictor
##
## Pre-processing: scaled (41), centered (41)
## Resampling: Cross-Validated (5 fold, repeated 1 times)
## Summary of sample sizes: 1551, 1550, 1550, 1550, 1551
## Resampling results across tuning parameters:
##
## size decay RMSE Rsquared MAE
## 2 0.01 1.085031 0.2162995 0.8225768
## 2 0.10 1.019160 0.2800496 0.7963020
## 4 0.01 1.189801 0.1685484 0.8824915
## 4 0.10 1.043455 0.2605620 0.8212750
## 6 0.01 1.094418 0.2417681 0.8641810
## 6 0.10 1.106200 0.2266146 0.8749263
##
## RMSE was used to select the optimal model using the smallest value.
## The final values used for the model were size = 2 and decay = 0.1.
# Hacer predicciones en el conjunto de prueba
test_results$nn <- predict(nn_tune, testing)
# Evaluar el modelo
nn_metrics <- postResample(pred = test_results$nn, obs = testing$average_grade)
print(nn_metrics)## RMSE Rsquared MAE
## 1.0631244 0.2662278 0.7906771
This last model predicts worse than Gradient Boosting.
Let’s put together the results obtained in this part.
# Crear un data frame con los resultados de cada modelo
model_comparison <- data.frame(
Model = c("KNN", "Random Forest", "Decision Tree",
"XGBoost", "Neural Network"),
RMSE = c(knn_metrics[1], rf_metrics[1], dt_metrics[1],
xgb_metrics[1], nn_metrics[1]),
R2 = c(knn_metrics[2], rf_metrics[2], dt_metrics[2],
xgb_metrics[2], nn_metrics[2]),
MAE = c(knn_metrics[3], rf_metrics[3], dt_metrics[3],
xgb_metrics[3], nn_metrics[3])
)
# Mostrar tabla de comparación
print(model_comparison)## Model RMSE R2 MAE
## 1 KNN 1.149084 0.1353695 0.8898548
## 2 Random Forest 1.009877 0.3317068 0.7651697
## 3 Decision Tree 1.121815 0.1720713 0.8738138
## 4 XGBoost 0.985138 0.3585118 0.7387951
## 5 Neural Network 1.063124 0.2662278 0.7906771
Ensamble model
To conclude our analysis, we will create an ensemble model combining the three best-performing models:Gradient Boosting, Random Forest and Stepwise Regression. The goal of this ensemble is to leverage the strengths of each model and see if the combination improves the overall prediction accuracy.
# Hacer predicciones individuales
test_results$stepwise <- predict(stepwise_model, testing)
test_results$rf <- predict(rf_tune, testing)
test_results$xgb <- predict(xgb_tune, testing)
# Hacer el ensemble promediando las predicciones
test_results$ensemble_avg <- (test_results$stepwise + test_results$rf +
test_results$xgb) / 3
# Evaluar el modelo ensemble
ensemble_metrics <- postResample(pred = test_results$ensemble_avg,
obs = testing$average_grade)
print(ensemble_metrics)## RMSE Rsquared MAE
## 0.9974109 0.3455710 0.7528790
The ensemble method, in this case, does not provide a major improvement. Gradient Boosting alone is better
As a final conclusion for this part we can say that it seems that the variables in the dataset do not fully explain the variability average grade. Our model’s performance has been limited in this second part, even if we tried several techniques, so maybe the features are not the most relevant or don’t capture the underlying factors that influence students’ average grade during their first year at Univesity.
Bibliography:
- Davis-Kean, P. E. (2005). The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment. Journal of Family Psychology, 19(2), 294–304.
- Fan, X., & Chen, M. (2001). Parental involvement and students’ academic achievement: A meta-analysis. Educational Psychology Review, 13(1), 1–22.
- Realinho, V., Vieira Martins, M., Machado, J., & Baptista, L. (2021). Predict Students’ Dropout and Academic Success [Dataset]. UCI Machine Learning Repository.